JP2009182570A - Image processor, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, an image processing method and an image processing program capable of performing an image restoration degree more satisfactorily. <P>SOLUTION: The image processor 1 for restoring a change image data i<SB>0</SB>includes a known image changing means for obtaining change known image data Simg' obtained by changing known image data Simg with image data contents known by change element information g, a change known image restoring means for restoring the change known image data Simg' to obtain restored known image data Si<SB>0 + n</SB>, a correction amount acquiring means for comparing the restored known image data Si<SB>0 + n</SB>with the known image data Simg and obtaining a correction amount d required for correcting the restored know image data Si<SB>0 + n</SB>to the known image data Simg, and a correcting means for correcting the restored image data i<SB>0 + n</SB>obtained by restoring the change image data i<SB>0</SB>on the basis of the correction amount d. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  Conventionally, it is known that image degradation occurs in a photographed image photographed with a camera or the like. Factors for this image degradation include camera shake during shooting, various aberrations of the optical system, lens distortion, and the like.

  In order to correct camera shake at the time of shooting, a method of moving a predetermined lens in the shooting lens and a method of processing by an electronic circuit are known. As a method for moving the lens, for example, a method is known in which camera shake is detected at the time of shooting, and a predetermined lens is moved in accordance with the detected camera shake, thereby suppressing deterioration of a captured image formed on the imaging unit. (See Patent Document 1).

  On the other hand, for example, as disclosed in Patent Document 2, a method of processing with an electronic circuit detects a change in the optical axis of a camera with an angular acceleration sensor, and detects a blurred state at the time of shooting from the detected angular velocity or the like. There is a method in which a transfer function is acquired, and the acquired transfer function is inversely transformed with respect to a captured image that has been deteriorated and imaged to restore the image.

JP-A-6-317824 (see abstract) Japanese Patent Laid-Open No. 11-24122 (see abstract)

  However, as disclosed in Patent Document 2, in the case of a method of obtaining a restored image by restoring a captured image that has been deteriorated and formed, the transfer function is affected by noise, blur information error, etc. For information that has become very small due to deterioration in the image, the restoration process will increase the error from the ideal captured image without any degradation that is desired to be obtained. It may not be possible.

  Therefore, the present invention increases the error from the ideal captured image without degradation by performing the restoration process even when the transfer function is affected by noise, blur information error, or the like. Therefore, an object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program that can perform image restoration more satisfactorily.

  In order to solve the above problem, in an image processing apparatus having a processing unit that restores changed image data, the processing unit changes the known image data whose content of the image data is known according to the change factor information. The amount of correction required to restore the restored known image data to the known image data by obtaining the restored known image data by restoring the data and the change known image data, and comparing the restored known image data with the known image data The restored image data obtained by restoring the changed image data is corrected based on the correction amount.

  By configuring the image processing apparatus in this way, it is possible to improve the degree of image restoration.

  In addition to the above-described invention, in another invention, the processing unit includes a known image changing unit that obtains changed known image data, a changed known image restoring unit that obtains restored known image data, and a correction amount obtaining unit that obtains a correction amount. And correction means for correcting the restored image data obtained by restoring the changed image data based on the correction amount.

  By configuring the image processing apparatus in this way, it is possible to improve the degree of image restoration.

  Further, in addition to the above-mentioned invention, the correction amount acquisition means performs a comparison between the restored known image data and the known image data in the frequency space, and obtains the correction amount as a phase amount or an amplitude amount. Is to correct the restored image data based on the correction amount obtained as the phase amount or the amplitude amount.

  By configuring the image processing apparatus in this way, it becomes easy to obtain the correction amount and correct the restored image.

  In another invention, in addition to the above-mentioned invention, the correction amount acquisition means compares the restored known image data and the known image data in the frequency space, and calculates the correction amount between the restored known image data and the known image data. Obtained as a difference vector, the correction means corrects the restored image data based on the correction amount obtained as the difference vector.

  By configuring the image processing apparatus in this way, the correction amount can be acquired with high accuracy, and the restored image can be corrected with high accuracy.

  In another invention, in addition to the above-described invention, the known image data is restored image data or changed image data.

  By configuring the image processing apparatus in this way, the correction amount can be obtained with high accuracy.

  According to another invention, in addition to the above-described invention, the known image data is an image in which restored image data or changed image data is edge-enhanced.

  By configuring the image processing apparatus in this way, the correction amount can be obtained with high accuracy.

  In order to solve the above-mentioned problem, in an image processing method for restoring changed image data, a known image changing step for obtaining changed known image data obtained by changing known image data whose contents are known according to change factor information; A step of restoring the known image data to obtain the restored known image data, and a step of restoring the known image data, comparing the restored known image data with the known image data, and correcting the restored known image data to the known image data. A correction amount acquisition step for obtaining a correction amount and a correction step for correcting the restored image data obtained by restoring the changed image data based on the correction amount are provided.

  By performing the image processing method in this way, the degree of image restoration can be improved.

  In order to solve the above problems, in an image processing program for restoring changed image data, known image change processing for obtaining changed known image data obtained by changing known image data whose contents are known by change factor information; Necessary for restoring the known image data by restoring the known image data by comparing the restored known image data with the known image data. The computer executes a correction amount acquisition process for obtaining a correction amount and a correction process for correcting the restored image data obtained by restoring the changed image data based on the correction amount.

  By installing an image processing program in a computer and causing the computer to execute the program, the degree of image restoration can be improved.

  According to the present invention, the degree of image restoration can be improved.

  Hereinafter, an image processing apparatus 1 according to a first embodiment of the present invention will be described with reference to the drawings. The image processing apparatus 1 is a so-called consumer digital camera that uses a CCD (Charge Coupled Device) for the image pickup unit 2. The present invention can also be applied to devices other than cameras, such as endoscope cameras and other imaging cameras, microscopes, binoculars, and diagnostic imaging apparatuses for nuclear magnetic resonance imaging. Note that the image processing method will be described together with the description of the image processing apparatus 1.

  The image processing apparatus 1 includes an imaging unit 2 that captures an image of a person or the like, a control system unit 3 that drives the imaging unit 2, and a processing unit 4 that processes an image captured by the imaging unit 2. Yes. In addition, the image processing apparatus 1 according to this embodiment further detects the recording unit 5 that records the image processed by the processing unit 4 and the change factor information that causes the change (degradation) of the captured image. It has a detection unit 6 and a change factor information storage unit 7 for storing known change factor information that causes an image change or the like.

  The imaging unit 2 includes a CCD, and converts a photographing optical system having a lens and light passing through the lens into an electric signal. In addition to the CCD, an imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) may be provided. The control system unit 3 controls each unit in the image processing apparatus 1 such as the imaging unit 2, the processing unit 4, the recording unit 5, the detection unit 6, and the change factor information storage unit 7.

  The processing unit 4 is configured by an image processing processor, and is configured by hardware such as an ASIC (Application Specific Integrated Circuit). The processing unit 4 includes a known image changing unit, a changed known image restoring unit, a correction amount acquiring unit, and a correcting unit, which will be described later, and a known image that is a base when generating a changed known image described later. May be stored.

  The processing unit 4 may be configured to process with software in addition to the hardware configured like ASIC. The recording unit 5 includes a semiconductor memory. However, a magnetic recording unit such as a hard disk drive, an optical recording unit using a DVD (Digital Versatile Disk), or the like may be employed.

  As shown in FIG. 2, the detection unit 6 includes two angular velocity sensors that detect the speeds around the X axis and the Y axis that are perpendicular to the Z axis that is the optical axis of the image processing apparatus 1. is there. Camera shake when shooting with the camera may cause movement in the X, Y, and Z directions and rotation around the Z axis, but the most affected by each variation is rotation around the Y axis. And rotation around the X axis. These two variations are only slightly varied, and the captured image is greatly blurred. For this reason, in this embodiment, only two angular velocity sensors around the X axis and the Y axis in FIG. 2 are arranged. However, for the sake of completeness, an angular velocity sensor around the Z axis may be further added, or a sensor for detecting movement in the X direction or the Y direction may be added. The sensor used may be an angular acceleration sensor instead of an angular velocity sensor.

  The change factor information storage unit 7 stores information about change factors that change the captured image and known change factor data that is known before the start of shooting. The information on the known change factor is, for example, aberration of the optical system. In this embodiment, the change factor information storage unit 7 stores information on aberrations of the optical system and lens distortion. However, when restoring blurring of camera shake described later, the information is Not used.

  Next, a processing method of the processing unit 4 of the image processing apparatus 1 configured as described above will be described.

  As will be described below, the processing unit 4 functionally implements an image restoration processing unit that executes a processing method for obtaining a restored image by treating image restoration as an optimization problem.

  Treating image restoration as an optimization problem means that “(1) the output for the input is uniquely determined”, “(2) If the output is the same, the input is the same”, “(3 The solution is converged by iteratively processing while updating the input so that the output becomes the same.

That is, as shown in FIG. 3A, the ideal image data “img”, which is the image data to be obtained, is changed to changed ideal image data “img ′” by the change factor information “g” that causes the image to change. Assuming that the change arbitrary image data “i ₀ ′” obtained by changing the arbitrary image data “i ₀ ” by the change factor information “g” approximates the changed ideal image data “img ′”. The arbitrary image data “i ₀ ” is updated as “i _{0 + n} ” (n is an integer of 1 or more), and the process of changing the arbitrary image data “i _{0 + n} ” with the change factor information “g” is repeated. In other words, any image data "i _{0 + n"} the change factor information data "g" change any image data obtained by changing the "i _{0 + n '"} is changed ideal image data "img' arbitrary image data such as approximate to""i if it is possible to obtain the _{0 + n} ", as shown in FIG. 3 (B), any image data as the original data for generating the change arbitrary image data" i _{0 + n ''} 'i _{0 + n} "is the ideal image data before the change" It can be said that this is a restored image approximated to “img”.

When the above-described processing is applied to an image captured by the imaging unit 2, the ideal image data “img” is captured image data (assuming that the subject can be captured in an ideal state without deterioration). Hereinafter, it is referred to as ideal captured image data, which is referred to as ideal captured image data “img”.), And the changed ideal image data “img ′” includes the ideal captured image data “img” and the change factor information “g”. ”And is equivalent to the actually captured image data (hereinafter referred to as actual captured image data, which will be referred to as actual captured image data“ img ′ ”). Therefore, the change arbitrary image data “i ₀ ′ (i _{0 + n} ′)” obtained by changing the arbitrary image data “i ₀ (i _{0 + n} )” with the change factor information “g” is the actual captured image data “img ′. ”, The arbitrary image data“ i ₀ (i _{0 + n} ) ”can be considered as image data that approximates the ideal photographed image data“ img ”.

  The operation of the image restoration processing means for realizing the above processing method will be described with reference to the flowchart shown in FIG.

In FIG. 4, “i ₀ ” is arbitrary initial image data stored in advance in the recording unit of the processing unit 4. “G” is data of change factor information (degradation factor information (point spread function which is a kind of transfer function)) detected by the detection unit 6, and is stored in the recording unit of the processing unit 4. “I ₀ ′” is comparison image data (changed arbitrary image data) obtained by changing the initial image data “i ₀ ” by the change factor information data “g”. “Img ′” is image data of a changed image to be restored, and here, is captured image data actually captured by the imaging unit 2, that is, actual captured image data.

“Δ” is difference data between the actual captured image data “img ′” and the comparison image data “i ₀ ′”. “K” is a feedback function representing a distribution ratio based on the change factor information data “g”. “I _{0 + n} ” (n is an integer equal to or greater than 1) is restored image data newly generated by distributing the difference data δ to the initial image data “i ₀ ” according to the distribution ratio “k”.

Here, the relationship between “img” and “img ′” is expressed by the following equation (1).

img '= img * g (1)

“*” Is an operator representing a superposition integral.

Note that the difference data “δ” may be a simple difference between corresponding pixels, but generally differs depending on the change factor information data “g” and is expressed by the following equation (2).

δ = f (img ′, img, g) (2)

The processing routine of the processing unit 4 (image restoration processing means) starts with preparing initial image data “i ₀ ” (step S101). As the initial image data “i ₀ ”, actual captured image data “img ′” may be used, and any image data such as black solid, white solid, gray solid, and checkered pattern may be used. . In step S102, data “i ₀ ” of an arbitrary image that is an initial image is input instead of “img” in the equation (1), and comparison image data “i ₀ ′” that is a degraded image is obtained. Next, the actual captured image data “img ′” is compared with the comparison image data “i ₀ ′”, and difference data “δ” is calculated (step S103).

Next, in step S104, it is determined whether or not the difference data “δ” is equal to or larger than a predetermined value. That is, the difference data “δ” is distributed to arbitrary initial image data “i ₀ ” based on the change factor information “g” to generate new restored image data “i _{0 + n} ”. Thereafter, steps S102, S103, S104, and S105 are repeated.

If the difference data “δ” is smaller than the predetermined value in step S104, the process ends (step S106). Then, the restored image data “i _{0 + n} ” at the time when the processing is completed is estimated as ideal photographed image data “img”.

The arbitrary initial image data “i ₀ ” is different from the ideal captured image data “img”, and the difference data “δ” is 0, that is, the actual captured image data “img ′” = restored image data “i”. It is difficult to set “ ₀ (i _{0 + n} )”, and in particular, when the change factor information “g” includes noise and error, it becomes more difficult to restore the difference data “δ” to 0. Therefore, when the difference data “δ” becomes smaller than the predetermined value, it is determined that the captured image has been restored to some extent, and the processing is terminated, and the restored image data “i ₀ (i _{0 + n} )” at this time Is the restored image data of the captured image.

  By the way, for example, assuming that the ideal photographed image data “img” is known image data whose contents are known, the known ideal photographed image data “img” is changed by the change factor information “ The changed ideal photographed image data “img ′” changed by “g” can be obtained as the change known image data. In the known image changing unit, the same processing as that in step S102 in the image restoration processing unit is performed.

Then, in the image restoration processing means that also functions as the change known image restoration means, the changed ideal captured image data “img ′” is set as arbitrary initial image data “i ₀ ”, and the processes from step S102 to step S105 are repeated. The restored image data “i _{0 + n} ” restored by the restoration process can be obtained as restored known image data.

Furthermore, by the correction amount acquisition unit, by comparing the restored image data as described above for recovery known image data "i _{0 + n"} known and ideal photographed image data "img", the restored image data "i _{0 + n"} known It is possible to obtain “d” as the correction amount data necessary to correct the ideal photographed image data “img”.

Assuming that the ideal photographed image data “img” is known, the correction amount data “d” obtained in this way is obtained by performing the above-described restoration process on any initial image data “i ₀ ”. The restored image data “i _{0 + n} ” of the obtained actual captured image data “img ′” is compared with the ideal captured image data “img”, and the restored image data “i _{0 + n} ” is converted into the known ideal captured image data “img”. This is considered to correspond to the correction amount necessary for correction. Accordingly, the restored image data “i _{0 + n} ” obtained by restoring the arbitrary initial image data “i ₀ ” is corrected with the correction amount data “d” in the correction unit, thereby allowing the arbitrary initial image to be corrected. The restored image data “i _{0 + n} ” obtained from the data “i ₀ ” can be used as restored image data in which the degree of restoration that is closer to the ideal photographed image data “img” is well corrected.

  In the above description, in order to make the concept of the correction amount data “d” easy to understand, the correction amount data “d” is obtained using the known image data as the ideal captured image data “img”. However, the ideal captured image data “img” is obtained. Is inherently unknown. On the other hand, it is considered that the correction amount data “d” can be obtained as a similar correction amount even if the known image data is arbitrary. Therefore, when the correction amount data “d” is actually obtained, any known image data is used.

However, it is preferable that the known image data for obtaining the correction amount data “d” is image data close to the ideal photographed image data “img” which is a restoration target. Therefore, it is preferable that the restored image data “i _{0 + n} ”, the actual captured image data “img ′”, or image data obtained by edge enhancement of these image data (“i _{0 + n} ”, “img ′”). By using such image data, highly accurate correction amount data “d” can be obtained.

  The above correction process will be described with reference to the flowchart of FIG.

First, known image data “Simg” whose contents are known is prepared (step S201). For example, restored image data “i _{0 + n} ”, actual captured image data “img ′”, or image data with edge enhancement of these image data (“i _{0 + n} ”, “img ′”) is prepared. Subsequently, in the known image changing means, change processing for changing the known image data “Simg” by the change factor information “g” is performed to generate changed known image data “Simg ′” (step S202). Then, in the change known image restoration means, the change known image data “Simg ′” is set as arbitrary initial image data “i ₀ ”, and the restoration process is repeated from step S102 to step S105 shown in FIG. Image data “Si _{0 + n} ” is generated (step S203).

Then, the correction amount acquisition unit, required for comparison with the restored known image data "Si 0 _{+ n"} and known image data "Simg", to correct the restored known image data "Si 0 _{+ n"} in a known image data "Simg" A correction amount acquisition process for obtaining correction amount data “d” is performed (step S204). Then, iterative restoration processing from step S102 to step S105 shown in FIG. 4 is performed on the photographed actual photographed image data “img ′” (step S205), and the restoration means obtained by the restoration processing in step S205 in the correction unit. The correction amount obtained in step S204 is corrected for the image data “i _{0 + n} ” to generate corrected restored image data “di _{0 + n} ” (step S206). The corrected restored image data “di _{0 + n} ” is corrected by the correction amount data “d” so that the restored image data “i _{0 + n} ” approaches the ideal photographed image data “img”. Note that the processing in step S205 may be performed before or in parallel with the processing in steps S201 to S204.

The calculation of the correction amount (step S204) and the correction of the restored image data “i _{0 + n} ” (step S206) can be performed in the frequency space as shown in FIGS. 6 and 7, for example. First, the correction amount calculation (step S204) will be described with reference to FIG.

The known image data “Simg” can be expressed as, for example, Is ↑ on the complex plane by being Fourier-transformed. Also, the change known image data “Simg ′” obtained by changing the known image data “Simg” by the change factor information “g” can be represented by, for example, Bs ↑ on the complex plane by Fourier transform. And Then, the restored image data “Si _{0 + n} ” generated by performing iterative restoration processing from step S102 to step S105 of FIG. 4 on the change known image data “Simg ′” (Bs ↑) For example, it is represented by Rs ↑. “↑” represents a vector. Similarly, in the following description, “↑” represents a vector.

As described above, when the restored image data “Si _{0 + n} ” and the known image data “Simg” are each Fourier-transformed and represented on the complex plane, each of these image data can be represented by amplitude and phase in the frequency space. Then, the correction amount data “d” for correcting Rs ↑ to Is ↑, the correction ratio M that is the ratio of the amplitudes of Rs ↑ and Is ↑, and the phase correction amount δθ that is the amount of phase shift are expressed by the equations (3), respectively. ) And equation (4).

M = | Is ↑ | / | Rs ↑ | (3)
δθ = θIs−θRs (4)

The correction amount data “d” is also present between the restored image data “i _{0 + n} ” that is restored image data for the actual captured image data “img ′” and the ideal captured image data “img”. it is conceivable that.

Therefore, corrected restored image data “di _{0 + n} ”, which is image data obtained by correcting the corrected image data “i _{0 + n} ” with the correction amount data “d” (correction ratio M and phase correction amount δθ), is restored image data. The image data is obtained as image data having a higher degree of restoration than “i _{0 + n} ”. This will be described with reference to FIG.

The actual captured image data “img ′” is represented by, for example, Br ↑ on the complex plane by being Fourier-transformed. The restored image data “i _{0 + n} ” generated by performing the iterative restoration process from step S102 to step S105 shown in FIG. 4 on the actual photographed image data “img ′” (Br ↑) is a complex plane by Fourier transform. In the above, for example, it is represented by Rr ↑. Then, the restored image data “i _{0 + n} ” (Rr ↑) is corrected by the correction amount data “d”, that is, the image data obtained by correcting the correction ratio M and the phase correction amount δθ is the corrected restored image data “di _{0 + n”.} As Ir ↑ on the complex plane. That is, the amplitude | Ir ↑ | and the phase θIr of the corrected restored image data “di _{0 + n} ” (Ir ↑) are obtained based on the following equations (5) and (6), respectively.

| Ir ↑ | = M | Rr ↑ | (5)
θIr = θRr + δθ (6)

Then, equation (5), a by inverse Fourier transform amplitude and phase of the correction restored image data "di 0 _{+ n"} (Ir ↑), corrected reconstructed image data in image space "di 0 _{+ n"} represented by (6) Can be sought. The corrected restored image data “di _{0 + n} ” is image data in which the degree of restoration closer to the ideal photographed image data “img” is higher than the restored image data “i _{0 + n} ”.

The method for obtaining the corrected restored image data “di _{0 + n} ” from the restored image data “i _{0 + n} ” may be as follows. As shown in FIG. 6, the correction amount data “d” for correcting Rs ↑ to Is ↑ is obtained as a correction ratio M and a phase correction amount δθ, and is obtained as a correction vector d ↑ for correcting Rs ↑ to Is ↑. Can do. Therefore, as shown in FIG. 8, the corrected restored image data “di _{0 + n} ” (Ir ↑) may be obtained by correcting the restored image data “i _{0 + n} ” (Rr ↑) with the correction vector d ↑. .

The above describes the example in which the correction processing is performed on the restored image data “i _{0 + n} ” obtained by the iterative restoration processing shown in FIG. 4. However, the inverse transformation of the transfer function is performed on the captured image data “img ′”. The above-described correction processing may be performed on the image data restored by performing the above, or the image data restored by a technique such as a Wiener filter or a general inverse filter to obtain the corrected restored image data “di _{0 + n} ”.

Here, a description will be given by taking, as an example, image data restored by performing inverse transformation of a transfer function. In this case, the relationship between the actual photographed image data “img ′” and the ideal photographed image data “img” is expressed as Equation (9) in the frequency space, where G is the transfer function.

Img ′ = Img × G (9)

Img ′: actual captured image data obtained by Fourier transforming img ′.
Img: ideal photographed image data obtained by Fourier transforming img.

Further, according to the following equation (10) obtained by modifying equation (9), the ideal captured image data “Img” can be obtained from the actual captured image data “Img ′”. That is, the ideal captured image data “Img” can be restored from the actual captured image data “Img ′”.

Img = Img ′ / G (10)

  However, as described above in the section “Background Art”, the transfer function is affected by noise, blur information error, etc., and as described in the section “Problems to be solved by the invention”, For information that has become very small due to deterioration, an error from an ideal photographed image without deterioration may be enlarged by performing a restoration process. Therefore, in many cases, the restored image obtained by restoring the actual captured image data “Img ′” according to Expression (10) is not restored to the ideal captured image data “Img”. In addition, for example, when panning is performed, the transfer function G becomes very small in the frequency space. In this case, the actual captured image data “Img ′” is restored by the equation (10). In many cases, the restored image is not restored to the ideal photographed image data “Img”. Therefore, the degree of restoration of the restored image can be increased by correcting the restored image obtained by Expression (10) as follows.

  First, in Formula (9), the ideal captured image data “Img” is arbitrarily known image data, and the change known image data “Img ′” obtained by changing the arbitrarily known image data “Img” with the transfer function G is represented as follows. Ask. Then, inverse transformation of the transfer function G is performed on the change known image data “Img ′” by the equation (10) to obtain restored known image data “Img”.

  Originally, the restored known image data “Img” restored by Expression (10) is the known image data “Img” that is the original image of the change known image data “Img ′” that is the object of inverse transformation. Should be equal to However, when there is a frequency band in which the transfer function G is very small in the frequency space due to the influence of noise or the like included in the transfer function, in the frequency band in which the actual captured image data has become very small. Actual photographed image data is divided by a very small transfer function G. Even if the mathematical expression is expressed as Expression (10), if it is calculated with an actual numerical value, the error becomes very large, and the restoration is known. The image data “Img” is not equal to the known image data “Img”. The correction amount necessary for correcting the restored known image data “Img” to the known image data “Img” is the ideal photographed image data obtained from the restored image data obtained by the equation (10) for the actual photographed image data “Img ′”. This is considered to correspond to a correction amount corrected to “Img”. Therefore, the degree of restoration of the restored image data can be increased even if the correction amount is corrected for the restored image data obtained by the equation (10) for the actual captured image data “Img ′”.

  In the description of the embodiment described above, the information stored in the change factor information storage unit 7 is not used. However, the known change factors stored here, such as optical aberrations and lens distortions, are not used. Such data may be used. In addition, the change factor information storage unit 7 may not be installed, and the image may be corrected or restored only by dynamic factors at the time of shooting, for example, shake.

  The image processing apparatus 1 according to the embodiment of the present invention has been described above, but various modifications can be made without departing from the gist of the present invention. For example, the processing performed by the processing unit 4 is configured by software, but may be configured by hardware composed of parts that perform part of the processing.

  Moreover, each processing method mentioned above may be programmed. Alternatively, the program may be stored in a storage medium, such as a CD, DVD, or USB memory, and read by a computer. In this case, the image processing apparatus 1 has reading means for reading a program in the storage medium. Further, the program may be stored in an external server of the image processing apparatus 1, downloaded as necessary, and used. In this case, the image processing apparatus 1 has communication means for downloading a program in the storage medium.

It is a block diagram which shows the main structures of the image processing apparatus which concerns on embodiment of this invention. It is an external appearance perspective view which shows the outline | summary of the image processing apparatus shown in FIG. 1, and is a figure for demonstrating the arrangement position of an angular velocity sensor. It is a figure for demonstrating the concept of the processing method performed with the image processing apparatus shown in FIG. It is a processing method (processing routine) performed by the processing unit of the image processing apparatus shown in FIG. 1 and is a flowchart for explaining the basic concept. 3 is a flowchart for explaining correction processing performed by the image processing apparatus shown in FIG. 1. It is a figure explaining the concept of calculation of the corrected amount performed by the image processing apparatus shown in FIG. 1, and correction | amendment of restored image data. It is a figure explaining the concept of calculation of the correction amount about the real picked-up image data performed by the image processing apparatus shown in FIG. 1, and correction | amendment of restored image data. It is a figure explaining the concept at the time of processing the calculation of the corrected amount about real picked-up image data performed by the image processing apparatus shown in FIG. 1, and correction | amendment of restoration image data with a vector of a difference.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 4 ... Processing part (Known image change means, change known image restoration means, correction amount acquisition means, correction means)
g ... Change factor information i ₀ ... Change image data i _{0 + n} ... Restored image data d ... Correction amount d ↑ ... Difference vector Simg ... Known image data Simg '... Change known Image data Si _{0 + n} ... Restored known image data

Claims (8)

In an image processing apparatus having a processing unit for restoring change image data,
The processing unit obtains the change known image data obtained by changing the known image data whose content of the image data is known by the change factor information, and the restoration known image data by performing the restoration process on the change known image data.
Comparing the restored known image data with the known image data to obtain a correction amount necessary to correct the restored known image data to the known image data,
A process of correcting the restored image data obtained by restoring the changed image data based on the correction amount is performed.
An image processing apparatus. The processor is
Known image changing means for obtaining the changed known image data;
Change known image restoration means for obtaining the restored known image data;
Correction amount obtaining means for obtaining the correction amount;
Correction means for correcting the restored image data obtained by restoring the changed image data based on the correction amount;
The image processing apparatus according to claim 1, further comprising: The correction amount acquisition means performs a comparison between the restored known image data and the known image data in a frequency space, obtains the correction amount as a phase amount or an amplitude amount,
The correction unit corrects the restored image data based on the correction amount obtained as the phase amount or the amplitude amount.
The image processing apparatus according to claim 2. The correction amount acquisition means performs a comparison between the restored known image data and the known image data in a frequency space, obtains the correction amount as a vector of a difference between the restored known image data and the known image data,
The correction unit corrects the restored image data based on the correction amount obtained as the difference vector.
The image processing apparatus according to claim 2.   The image processing apparatus according to claim 1, wherein the known image data is the restored image data or the changed image data.   The image processing apparatus according to claim 1, wherein the known image data is an image obtained by edge-enhancing the restored image data or the changed image data. In an image processing method for restoring change image data,
A known image change step for obtaining change known image data obtained by changing known image data whose content of image data is known by change factor information;
A change known image restoration step of obtaining the restoration known image data by performing the restoration processing of the change known image data;
A correction amount obtaining step for obtaining a correction amount necessary to correct the restored known image data to the known image data by comparing the restored known image data with the known image data;
A correction step of correcting the restored image data obtained by restoring the changed image data based on the correction amount;
An image processing method comprising: In an image processing program for restoring change image data,
Known image change processing for obtaining change known image data obtained by changing known image data whose content of image data is known by change factor information; and
A change known image restoration process for obtaining the restored known image data by performing the restoration process on the change known image data;
A correction amount acquisition process for obtaining a correction amount necessary for correcting the restored known image data to the known image data by comparing the restored known image data with the known image data;
A correction process for correcting the restored image data obtained by restoring the changed image data based on the correction amount;
An image processing program for causing a computer to execute.

Images