JPWO2007122883A1 - Signal processing device - Google Patents

Abstract

In restoring a signal that has changed due to deterioration or the like, it is possible to prevent an increase in the size of the apparatus and to provide a realistic signal processing apparatus. The signal processing apparatus 1 includes a processing unit 4 that restores original signal data, such as a signal before change, from original signal data (hereinafter referred to as change data) in which a change such as deterioration has occurred. Then, a change data area in which change data is stored and a restoration data area in which the data of the restored signal is stored for each restoration process are provided, and the processing unit repeats the process for obtaining the original signal When the energy value of the remaining data becomes less than zero in the process of the iterative process, the energy value of the remaining data is converted to a part of the energy already transferred to the restored data area by using the change factor information. The repetitive process is performed while performing the process of returning to the change data area so as to become zero or more, and the data formed in the restored data area at the end of the repetitive process is used as the original signal data.

Description

  The present invention relates to a signal processing apparatus.

  Conventionally, it has been known that when an image is processed by an image processing apparatus such as a camera, the image sometimes deteriorates. Factors of image degradation include camera shake during shooting, various aberrations of the optical system, lens distortion, and the like.

  In order to correct camera shake during shooting, a method of moving a lens and a method of circuit processing are known. For example, as a method of moving a lens, a method of detecting camera shake of a camera and correcting it by moving a predetermined lens in accordance with the detected camera shake is known (see Patent Document 1). In addition, as a circuit processing method, a change in the optical axis of the camera is detected by an angular acceleration sensor, a transfer function indicating a blurring state at the time of shooting is acquired from the detected angular velocity, etc., and the acquired transfer function is obtained for a shot image. Is known that performs inverse transformation of the above to restore the image (see Patent Document 2).

  In addition to general captured images, it is known that various signals such as voices, X-ray photographs, microscopic images, and seismic waveforms are deteriorated or changed due to blurring or other causes.

JP-A-6-317824 (see abstract) Japanese Patent Laid-Open No. 11-24122 (see abstract)

  The camera employing the camera shake correction described in Patent Document 1 requires a hardware space for driving a lens, such as a motor, and becomes large. In addition, such hardware itself and a drive circuit for operating the hardware are necessary, which increases costs. In addition, in the case of camera shake correction described in Patent Document 2, the above-described problems are eliminated, but there are the following problems. That is, it is theoretically possible that image restoration is performed by inverse transformation of the acquired transfer function, but as a practical problem, image restoration is difficult for the following two reasons.

  First, the transfer function to be acquired is very weak against noise, blur information error, etc., and the value fluctuates greatly due to these slight fluctuations. For this reason, the corrected image obtained by the inverse transformation is far from an image photographed with no camera shake, and cannot be used in practice. Second, when performing inverse transformation considering noise, etc., a method of estimating the solution by singular value decomposition etc. of the solution of simultaneous equations can be adopted, but the calculated value for the estimation becomes astronomical size. In practice, there is a high risk of being unable to solve.

  The above-mentioned problems that occur in images also appear in various general data, and it is assumed that the restoration of the signal by the inverse transformation of the transfer function is accurate as well as the case where the acquired transfer function is inaccurate. Even it has become difficult. In addition, obtaining a transfer function that is 100% accurate is a situation that is not possible in the natural world.

  As described above, an object of the present invention is to provide a signal processing apparatus that has a realistic circuit processing method while preventing an increase in size of the apparatus when restoring a signal.

  In order to solve the above-described problems, the signal processing apparatus of the present invention should have been acquired from the original signal data (hereinafter referred to as “change data”) that has undergone changes such as deterioration, or the signal before the change. A processing unit for restoring signals or their approximate signal data (hereinafter referred to as original signal data), a change data area in which change data is stored, and a signal of the restored signal for each restoration process A restoration data area for storing data (hereinafter referred to as restoration data) is provided, and the processing unit uses the change factor information data that causes the change to restore the restored data from the change data area. Move to the area, generate restoration data, replace the remaining data of the change data area remaining by the transfer with the change data, and repeat the same process If the energy value of the remaining data becomes less than zero during the iterative process, the energy value of the remaining data becomes zero or more using the change factor information for a part of the energy that has already moved to the restored data area. In this manner, the process is repeated while returning to the changed data area, and the data formed in the restored data area at the end of the repeated process is used as the original signal data.

  According to the present invention, since it is assumed that the original signal data has changed to change data in accordance with the change factor information data, the energy of the change data is restored by using the change factor information data serving as the same filter. By shifting to the area, the original signal data is reliably restored as restored data in the restored data area. Also. As a result of the energy transfer, it is possible to avoid a theoretically impossible situation where the energy value of the remaining data is less than zero, so that the restoration accuracy of the restoration data can be improved. Here, the change data stored in the change data area may be obtained by processing the change data while keeping the state of energy constituting the change data as it is (the same applies hereinafter). The restoration data stored in the restoration data area may be data obtained by processing the restoration data while keeping the state of energy constituting the restoration data as it is (hereinafter the same). Furthermore, “migrating” means moving the energy value literally from the changed data area to the restored data area, and removing that energy from the changed data area and creating new energy in the restored data area. (Including the same). Furthermore, the change data area and the restored data area include both temporarily formed areas or permanently formed areas (the same applies hereinafter).

  In another invention, in addition to the above-described invention, when the energy value of the remaining data is less than zero, the value is processed to be zero or more. If the energy value of the remaining data becomes zero, it can be considered that the restoration accuracy of the restoration data restored from the change data is very good.

  In another invention, in addition to the above-described invention, at the time of repetition processing, the energy that shifts to the restoration data area is added to the restoration data already stored in the restoration data area at each repetition, and the energy value of the remaining data Is processed to approach zero in the range of zero or more. If the remaining data approaches zero in the range of zero or more, most of the energy in the change data area shifts to the restoration data area, so that the restoration data approaches the original signal data.

  In order to solve the above problems, another signal processing apparatus of the present invention has a processing unit that restores original signal data consisting of a plurality of elements from change data consisting of a plurality of elements, and the change data is stored. A change data area and a restoration data area for storing the restored signal data (hereinafter referred to as restoration data) for each restoration process are provided, and the processing unit is an element in one element of the change data. Using the centroid value of the response characteristic function of the change factor information data that causes the change, the energy is transferred from the change data area to the restoration data area, and the element energy corresponding to the transferred element energy is changed. Perform the process of excluding from the data area using the change factor information data, and also perform the process for this one element sequentially for the other elements to restore to the restored data area Data is generated, the remaining data of the changed data area remaining by exclusion is replaced with the changed data, and the same process is repeated for each element, and the element energy transferred to the restored data area is added to the restored data each time it is repeated. In the course of these series of processing, if any element energy value of the remaining data becomes less than zero in the course of these series of processing, the element that has already moved to the restoration data area Using a change factor information, a part of the energy is returned to the change data area so that the element energy value that is less than zero is greater than or equal to zero. A process of approaching zero in the range is performed, and the restored data formed in the restored data area at the end of the process is used as the original signal data.

  According to the present invention, since it is assumed that the data of the original signal has changed to change data according to the change factor information data, the centroid value (energy is the energy of the response characteristic function, which is the change factor information data to be the same filter). By using the reciprocal of the value of the most concentrated part), much of the energy of the change data is transferred to the restoration data area, and the element energy corresponding to the transferred energy is transferred from the change data area to the change factor information data. If the energy value of the remaining data (hereinafter referred to as the remaining energy amount) can be made close to zero by excluding it by use, the original signal data is reliably restored as restored data in the restored data area. In addition, as a result of the energy transfer, it is possible to avoid a situation that cannot theoretically occur because the energy value of any element constituting the remaining data is less than zero, so that the restoration accuracy of the restoration data can be improved. it can. The response characteristic function includes an impulse response function, a unit response function, and the like. When the signal data is image data, the impulse response function is a point spread function.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit, when generating the restored data, if the energy value of the remaining data becomes less than or equal to a predetermined value within a range of zero or more, Processing to stop. When this configuration is adopted, the processing is stopped even if the remaining energy amount does not become “0”, so that it is possible to prevent the processing from being prolonged. Further, since the value is less than or equal to the predetermined value, the restored data to be approximated is closer to the original signal data before the change (before deterioration or the like) that is the source of the change data. Furthermore, when there is noise or the like, there is a tendency that the remaining energy amount cannot be “0” in reality, but even in such a case, the process is repeated infinitely. Must not.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit performs a process of stopping when the number of times of generating the restoration data reaches a predetermined number when generating the restoration data. When this configuration is adopted, the process is stopped regardless of whether the remaining energy amount becomes “0”, so that it is possible to prevent the process from being prolonged. Further, since the processing is continued up to a predetermined number of times, the restored data to be approximated becomes closer to the original signal data before the deterioration that becomes the source of the change data. In addition, when there is noise or the like, a situation where the remaining energy amount does not become “0” is likely to occur in reality, but even in such a case, it is terminated in a predetermined number of times, so it is processed infinitely. Will not be repeated.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit, when generating the restoration data, has an energy value of zero or more when the number of times the restoration data is generated reaches a predetermined number. When the value is less than or equal to a predetermined value within the range, the process is stopped. In this invention, since the number of times of processing and the remaining energy amount are combined, the restoration accuracy is better than when the number of processing times is simply limited or when the remaining energy amount is limited. It can be set as the process which balanced the shortness of the processing time.

  In order to solve the above problems, another signal processing apparatus of the present invention has a processing unit that restores original signal data consisting of a plurality of elements from change data consisting of a plurality of elements, and the change data is stored. A change data area and a restoration data area in which data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process are provided, and the processing unit has energy in one element of the change data. Is transferred from the change data area to the restoration data area using the change factor information data that causes the change, and the energy corresponding to the transferred energy is transferred from the change data area using the change factor information data. Perform the process to exclude, and also perform the process for one element for all the other elements in sequence, generate the restoration data in the restoration data area, and remain by the migration If the value of the remaining data in the digitized data area is less than or equal to a predetermined value in the range of zero or more or smaller than the predetermined value, the processing is stopped, the restored data is treated as original signal data, If the value is greater than or equal to the value, the remaining data is replaced with the change data, and the same process is repeated, and each time the process is repeated, the element energy that is transferred to the restored data area is added to the restored data to generate new restored data. If any element energy value of the remaining data is less than zero, an element that is less than zero in the remaining data by using a part of the element energy that has already been transferred to the restored data area using the change factor information The remaining data amount is compared with a predetermined value while performing a process of returning to the change data area so that the energy value becomes zero or more.

  According to the present invention, since it is assumed that the data of the original signal has changed to change data according to the change factor information data, the energy of the change data is reduced by using the change factor information data that is the same filter. If the data can be transferred to the restored data area, the original signal data is reliably restored in the restored data area. In addition, as a result of the energy transfer, it is possible to avoid a situation that cannot theoretically occur because the energy value of any element constituting the remaining data is less than zero, so that the restoration accuracy of the restoration data can be improved. it can. Further, the energy is transferred from the change data area to the restoration data area using the change factor information, and the remaining portion is only obtained when the remaining energy amount value in each element of the change data area exceeds the predetermined value or exceeds the predetermined value. Since the same process is repeated by replacing the data with the change data, the restoration process can be performed quickly. Furthermore, since restoration processing is performed by transferring energy from the change data area to the restoration data area, there is almost no increase in hardware, and the apparatus does not increase in size. For this reason, a signal processing apparatus having a realistic circuit processing method can be provided for signal restoration.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit performs a process of stopping when the number of times of generating the restoration data reaches a predetermined number when generating the restoration data. When this configuration is adopted, the process is stopped regardless of whether the remaining energy amount becomes “0”, so that it is possible to prevent the process from being prolonged. Further, since the processing is continued up to a predetermined number of times, the restored data is closer to the original signal data. Furthermore, when there is noise or the like, a situation in which the remaining energy amount does not become “0” is likely to occur in reality, but in such a case, the processing is repeated infinitely, but this configuration is If adopted, such a problem does not occur.

  In addition to the above-described invention, the signal processing apparatus according to another invention may generate one or more of the maximum value, the average value, or the total value of the remaining data values of each element when generating the restored data. Comparison with a predetermined value is performed. When this configuration is adopted, the remaining energy amount in each element constituting the change data can be made close to zero, so that the degree of approximation between the restored data and the original signal data can be further increased.

  In the signal processing device according to another invention, in addition to the above-described invention, in the returning process, when the restoration data is generated once or a plurality of times, any element energy value of the remaining data becomes less than zero. The target energy is the element energy that has been transferred to the restoration data area before that time. When this configuration is used, the energy restoration results in an energy value that is less than zero in any element of the remaining data, so it is possible to avoid a situation that could not occur theoretically. Can be made.

  In addition to the above-described invention, a signal processing apparatus according to another invention uses signal data as image data. As a result, even if image degradation occurs due to camera shake, an original image that has undergone degradation or the like, an image that should have been originally captured, or an approximate image thereof (hereinafter referred to as an original image). ) Can be restored.

  According to the present invention, when restoring a signal that has changed due to deterioration or the like, it is possible to prevent an increase in the size of the apparatus and to provide a realistic signal processing apparatus.

It is a block diagram which shows the main structures of the signal processing apparatus which concerns on embodiment of this invention. It is an external appearance perspective view which shows the outline | summary of the signal processing apparatus shown in FIG. 1, and is a figure for demonstrating the arrangement position of an angular velocity sensor. FIG. 2 is a processing flowchart for explaining a processing routine according to an image restoration processing method (repetitive processing) performed by a processing unit of the signal processing device shown in FIG. 1. It is a figure for demonstrating the concept of the processing method performed with the process part of the signal processing apparatus shown in FIG. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 3 using camera shake as an example, and is a table showing the concentration of pixel energy when there is no camera shake. It is a figure for demonstrating concretely the processing method shown in FIG. 3 taking an example of camera shake, and is a figure which shows image data when there is no camera shake. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 3 using camera shake as an example, and is a diagram showing dispersion of pixel energy when camera shake occurs. It is a figure for demonstrating an example of the condition of camera shake when there existed dispersion | distribution of the pixel energy shown in FIG. It is a figure explaining the processing method shown in FIG. 3 taking the image data shown in FIG. 8 deteriorated by camera shake as an example. It is a conceptual diagram which shows the idea of the review method of the transition value in the transition process at the time of performing the processing method shown in FIG. FIG. 9 is a diagram showing a state of processing of the image data shown in FIG. 8 by the processing method in FIG. 3 and a state of processing at the stage of “n = 2”. FIG. 9 is a diagram showing a state of processing of the image data shown in FIG. 8 by the processing method in FIG. 3 and a state of processing at the stage of “n = 3”. It is a figure for demonstrating an example of the review method 2 of the transition value in the migration process at the time of performing the process shown in FIG. It is a figure for demonstrating an example of the review method 3 of the transition value in the migration process at the time of performing the process shown in FIG. It is a figure for demonstrating an example of the review method 4 of the transfer value in the transfer process at the time of performing the process shown in FIG. FIG. 4 is a processing flowchart for explaining a processing routine according to a method for reviewing transition values when the processing method shown in FIG. 3 is executed.

Explanation of symbols

1 Signal Processing Device 2 Receiver (Photographing Unit)
3 Control System 4 Processing Unit 5 Recording Unit 6 Detection Unit 7 Factor Information Storage Unit G Change factor information data (degradation factor information data)
Ga Centroid value of point spread function Img 'Original image data (captured image)
Img Original correct image data without deterioration (original image)
E Original image pixel energy F _n transition pixel energy R _n restoration data cE correction energy E _n remaining energy amount (energy to be processed)
Pixel energy cE correction energy removed from the predetermined value T _n original image data area of the X balance energy amount

  Hereinafter, a signal processing device 1 according to an embodiment of the present invention will be described with reference to the drawings. The signal processing device 1 is an image processing device and is used as a consumer camera. However, the signal processing device 1 is a camera for surveillance, a television camera, a handy type video camera, an endoscope camera, and the like. It can be applied to devices other than cameras, such as cameras for applications, microscopes, binoculars, and diagnostic imaging apparatuses such as NMR imaging.

  FIG. 1 shows an outline of the configuration of the signal processing apparatus 1. The signal processing device 1 includes an imaging unit 2 that captures an image of a person, a control system unit 3 that drives the imaging unit 2, and a processing unit 4 that processes an image captured by the imaging unit 2. ing. The signal processing apparatus 1 according to this embodiment further includes a recording unit 5 that records an image processed by the processing unit 4 and an angular velocity sensor, and detects change factor information that causes a change such as image degradation. And a factor information storage unit 7 for storing known change factor information that causes image degradation and the like. Note that when the signal processing device 1 is applied as a device other than an image processing device, the photographing unit 2 is a receiving unit 2 that receives various input signals such as an audio signal (hereinafter, the photographing unit 2 and the receiving unit as appropriate). 2).

  The photographing unit 2 includes a photographing optical system having a lens and a photographing element such as a CCD or C-MOS that converts light passing through the lens into an electric signal. The control system unit 3 controls each unit in the signal processing device 1 such as the photographing unit 2, the processing unit 4, the recording unit 5, the detection unit 6, and the factor information storage unit 7.

  The processing unit 4 is configured by an image processing processor, and is configured by hardware such as an ASIC (Application Specific Integrated Circuit). The processing unit 4 generates a sampling frequency for detecting vibrations such as camera shake to be detected and supplies the sampling frequency to the detection unit 6. The processing unit 4 controls the start and end of vibration detection. When the signal processing device 1 is applied as a device other than the image processing device, the reception sensitivity of the receiving unit 2 can be changed according to the magnitude of the input signal or the like.

Further, the processing unit 4 is not configured as hardware such as an ASIC, but may be configured to perform processing by software. In the processing unit 4, an original image data area serving as a change data area and a restored image data area serving as a restored data area are permanently arranged. Further, the processing unit 4 stores a maximum value “E _max ” of the remaining energy amount of each pixel described later. The recording unit 5 is composed of a semiconductor memory, but magnetic recording means such as a hard disk drive or optical recording means using a DVD or the like may be employed. Note that a change data area and a restoration data area may be provided in the recording unit 5, and the maximum value “E _max ” of the remaining energy amount may be stored.

  As shown in FIG. 2, the detection unit 6 includes two angular velocity sensors that detect the speeds around the X axis and the Y axis that are perpendicular to the Z axis that is the optical axis of the signal processing device 1. is there. By the way, camera shake at the time of photographing with the camera also causes movement in each direction of the X direction, Y direction, and Z direction, and rotation around the Z axis, but the greatest influence by each variation is around the Y axis. Rotation and rotation around the X axis. These two variations are only slightly varied, and the captured image is greatly blurred. For this reason, in this embodiment, only two angular velocity sensors around the X axis and the Y axis in FIG. 2 are arranged. However, for the sake of completeness, an angular velocity sensor around the Z axis may be further added, or a sensor for detecting movement in the X direction or the Y direction may be added. The sensor used may be an angular acceleration sensor instead of an angular velocity sensor.

  The factor information storage unit 7 is a recording unit that stores change factor information such as known deterioration factor information, such as a point spread function calculated based on aberrations of the optical system and / or detected vibrations. The point spread function recorded by the factor information storage unit 7 is used by the processing unit 4 at the time of restoration processing of an original image, for example, an image having undergone a change such as degradation that has been taken immediately after the calculation. Here, when the original image restoration process is executed, the original image is taken when the imaging power is turned off, when the processing unit 4 is not operating, or when the operating rate of the processing unit 4 is low. It can be a period delayed from In this case, the original image data stored in the recording unit 5 and the change factor information such as the point spread function for the original image stored in the factor information storage unit 7 are associated with each other. Stored for a long time. As described above, the advantage of delaying the time for executing the restoration process of the original image from the time of shooting the original image is that the burden on the processing unit 4 at the time of shooting involving various processes can be reduced. When the signal processing device 1 is applied as a device other than an image processing device, the temperature, humidity, etc. detected by the detection unit 6 may change the reception characteristics of the reception unit 2 and the characteristics of the entire system. They can be recorded and used as change factor information. Further, the response characteristic function of the system that is known in advance, such as the impulse response of the system, can be stored in the factor information storage unit 7.

  Next, an outline of an example of an image restoration processing method performed by the processing unit 4 of the image processing apparatus which is the signal processing apparatus 1 configured as described above will be described with reference to FIG.

In FIG. 3, “E” is light energy (original image pixel energy) of each pixel of original image data Img ′ (details will be described later) in which a change such as deterioration has occurred, and becomes a change data area. Stored in the original image data area. “F _n ” is the pixel energy (hereinafter referred to as “transition pixel energy”) transferred from the original image data area that is the nth change area to the restored image data area that is the restored data area. “E _n ” is the remaining energy amount remaining in the original image data area which is the changed data area due to the transition of the transition energy F _n from the first time to the n-th time, and is the energy to be processed. “R _n ” is restored data stored in a restored image data area serving as a restored data area, and becomes approximate data of the original image data “Img” by performing the image restoration process shown in FIG. "X" is a predetermined value of the balance energy amount E _n. “G” is data of change factor information (= deterioration factor information (point image function)) detected by the detection unit 6 and is stored in the recording unit of the processing unit 4. “Ga” is the barycentric value of the point image function of the data of the change factor information G (= degradation factor information (point image function)) detected by the detection unit 6. “Img” is data of an original image, that is, an image that should have been originally taken, and is original signal data. Note that the original signal data is defined in this specification as including the signals before the change and the data of the approximate signals in addition to the original image Img (the signal that should have been originally acquired). The original image data Img ′ indicates data of a captured image, that is, a deteriorated image. Here, it is assumed that the relationship between Img and Img ′ is expressed by the following equation (1).
Img ′ = Img * G (1)
Here, “*” is an operator representing a superposition integral.

Further, “T _n ” illustrated in FIG. 3 is a pixel energy amount for removing pixel energy corresponding to the transition pixel energy F _n from the original image data region, and is the same amount as the transition pixel energy F _n . “E _max ” is the maximum value of the remaining energy amount of each pixel in the original image data area. Here, relationship between the transition pixel energy F _n balance energy amount E _n of each pixel in the original image data area is to be expressed by the following equation (2). That is, the transition pixel energy F _n is obtained by using the reciprocal number of the barycentric value Ga of the point spread function which is the data G of the change factor information, as the remaining energy amount E _n (energy to be processed) of each pixel in the original image data area. Obtainable. Here, “k” is a component ratio with respect to the energy of a pixel corresponding to the specific pixel in the original image Img, which is included in the energy of the specific pixel of the captured original image data Img ′. Since “k” is unknown, it can be arbitrarily set within the range of “0 <k ≦ 1”.
F _n = k × E _n / Ga (2)

The processing routine of the processing unit 4 in FIG. 3 starts by extracting the pixel energy of one element constituting the original image data Img ′ as the original image pixel energy E (step S101). Here, at the present stage (n = 0), E ₀ = E (original image data), and the restored data R ₀ of the restored image data area is 0. In the first processing (n = 1), first, the transition pixel energy F is obtained by multiplying the original image pixel energy E = E ₀ by the inverse of the barycentric value Ga of the point spread function (the largest value of the change factor information data G). ₁ is obtained (step S102). Next, the transfer pixel energy F ₁ is transferred to the restored data R ₁ in the restored image data area. That is, by adding the transition pixel energy _{F 1} to restore the data _{R 0} of the restored image data region is first restored data _{R 1} (step S103).

Next, the pixel energy of the transition pixel energy F ₁ shifted to the restored image data area is removed from the original image data area. In that case, the point spread function which is the change factor information data G is used. This is because the deterioration of data is caused by passing through a filter called change factor information data G, and the transition pixel energy F _n can be removed from the original image data area so that there is no contradiction before and after the deterioration. It is. Therefore, the pixel energy amount T ₁ obtained by the superposition integration (step S104) of the transition pixel energy F ₁ and the change factor information data G is removed from the original image data area, and the remaining energy of the original image data area as the residual energy is removed. obtaining a quantity _{E 1} (step S105).

The above steps S102, S103, S104, and S105 are sequentially performed for all the remaining pixels constituting the original image data Img ′ in the original image data area. At this time, the energy E ₀ to be processed in step S102 is an amount corresponding to the pixel obtained by superposition integration when the transition pixel energy F _n is removed from the original image data area in the energy transition process of the surrounding pixels. Is the value removed. Then, for all the pixels, the above steps S102, S103, after S104 and S105 has been performed, stores the maximum value _{E max} of balance energy amount _{E 1} of the original image data area (step S106).

Next, whether or not the maximum value E _max of the remaining energy amount E ₁ is less than the predetermined value X, that is, most of the original image pixel energy E = E ₀ originally present in the original image data area is transferred to the restored image data area and restored. restoring data R ₁ in the image data area to perform determination of whether it can be regarded as the original image data Img (step S107). In this example, the predetermined value X is set to a value close to “0” other than “0”, it is determined whether E _max is less than the predetermined value X, and when it is equal to or larger than the predetermined value X, “n = n + 1” = 2 and the processes of steps S102, S103, S104, S105, S106, and S107 are performed again for all pixels. In the processing procedure of FIG. 3, “n = n + 1” is performed when steps S <b> 102, S <b> 103, S <b> 104, and S <b> 105 are finished for all pixels and steps S <b> 106 and S <b> 107 are performed. That is, the number of processes for the entire image is represented as n. Note that step S107 may be a step of determination of whether a pixel of the balance energy amount E _n equal to or greater than a predetermined value X number is. In this case, even if there are some pixels which is the balance energy amount E _n equal to or greater than a predetermined value X, the restored data R _n The fewer the number can be regarded as being sufficiently approximated to the original image data Img .

A process when the maximum value E _max of the remaining energy amount is equal to or greater than the predetermined value X in step S107 will be described. Since E _max is equal to or greater than the predetermined value X, n = n + 1 is set, and the processes of steps S102, S103, S104, S105, S106, and S107 are performed for all pixels. That is, the transition pixel energy F _n is obtained by multiplying the remaining energy E _n−1 by the inverse of the barycentric value Ga of the point spread function (the largest value of the change factor information data G) (step S102). Next, the transfer pixel energy F _n is transferred to the restored image data area, and new restored data R _n is obtained. That is, the current transition pixel energy F _n is added to the restored data R _n−1 of the restored image data area restored up to the previous (n−1) to obtain new restored data R _n . Then, the pixel energy T _n obtained in the convolution (step S104) and shifts the pixel energy F _n and the change factor information data G is removed from the original image data area, obtain a new balance energy E _n (Step S105) .

The above steps S102, S103, S104 and S105 are sequentially performed for all remaining pixels in the original image data area. Then, the remaining energy amount maximum value “E _max ” in each pixel is stored (step S106). Then, it is determined whether or not the remaining energy amount maximum value E _max is less than a predetermined value X (step S107). If E _max is equal to or greater than the predetermined value X, “n = n + 1” is set, and the processes of steps S102, S103, S104, S105, S106, and S107 are repeated. If E _max is less than the predetermined value X, it is determined that the restored data R _n can be sufficiently approximated to the original image data Img, and the restoration process is terminated (step S108).

Based on FIG. 4, the concept of the camera shake restoration process according to this embodiment will be described below. If the original image data Img is changed to the original image data Img ′ by the change factor information data G, the original image data Img ′ is constructed by using the change factor information data G that is the same filter. If all of the original image pixel energy E in all the pixels is transferred to the restored image data area, the restored data R _n in the restored image data area should theoretically approach the original image data Img.

  Next, details of the camera shake restoration processing method (repetition processing of steps S102, S103, S104, S105, S106, and S107) shown in FIGS. 3 and 4 will be described in detail with reference to FIGS. 9 will be described.

(Restore algorithm)
When there is no image degradation due to camera shake or the like, light energy (pixel energy) corresponding to a predetermined pixel is concentrated on that pixel during the exposure time. In addition, when there is camera shake, the pixel energy is distributed to the pixels that are shaken during the exposure time. Further, if the blur during the exposure time is known, it is possible to know how to disperse the pixel energy during the exposure time, so that it is possible to create a blur-free image from the blurred image.

  Hereinafter, for the sake of simplicity, the description will be made in one horizontal dimension. Let the pixels be S-1, S, S + 1, S + 2, S + 3,... In order from the left, and pay attention to a certain pixel S. When there is no image degradation due to blurring or the like, the pixel energy during the exposure time is concentrated on the pixel, and therefore the degree of concentration of the pixel energy is “1.0”. This state is shown in FIG. Further, as shown in FIG. 5, an example of the photographing result when there is no image deterioration is shown in the table of FIG. What is shown in FIG. 6 is the correct image data Img when no deterioration occurs. Each data is represented by 8-bit (0 to 255) data.

  There is image degradation due to blur etc. during the exposure time. 50% of the exposure time is blurred to the Sth pixel, 30% to the S + 1th pixel, and 20% to the S + 2th pixel. Suppose that The method of distributing the pixel energy is as shown in the table of FIG. This becomes the data G of the change factor information. Further, since the barycentric value Ga of the point spread function is a value of a portion where energy is most concentrated, it is a value “0.5” of a portion that has been shifted by 50% of the exposure time.

  The blur is uniform in all pixels, there is no upper blur (vertical blur), and if the blur is in the range of three pixels as shown in FIG. 7, the blur situation, that is, the pixel energy distribution status of each pixel is Specifically, as shown in FIG. 8, for example, “120” of the pixel “S-3” is “α = 0.5”, “β = 0. 3 ”and“ γ = 0.2 ”(FIG. 7),“ 60 ”is applied to the“ S-3 ”pixel,“ 36 ”is applied to the“ S-2 ”pixel, and“ S-1 ”is applied to the pixel. Disperse as “24”. Similarly, “105” which is the pixel energy of “S-2” is set to “52.5” for “S-2”, “31.5” for “S-1”, and “21” for “S”. scatter. Similarly, the pixel energy is dispersed in the other pixels.

The relationship between “A”, which is the energy of a specific pixel in the original image data Img, and energy “A ′” of a pixel corresponding to the specific pixel in the captured original image Img ′ is generally expressed as follows: It is represented by the formula (3).
A × Ga = k × A ′ (3)
Since the barycentric value “Ga” of the point spread function is a value of a portion where energy is most concentrated, in this example, as shown in FIG. 7, “α = 0.5”. In this “α”, the energy “A” of a specific pixel in the original image data Img is dispersed into the energy “A ′” of a pixel corresponding to the specific pixel in the captured original image Img ′ due to blurring. “A ′” is dispersed from energy of other pixels in the vicinity of “A” in addition to “A × α” by a dispersion ratio other than α (β, γ, etc. in FIG. 7). Consists of the sum of pixel energy. Needless to say, the equation (3) can be applied even if the α value is other than “α = 0.5” (FIG. 7). The expression (3) can also be applied to the case where blurring occurs over a range less than or exceeding the range of three pixels as shown in FIG. Furthermore, even when the barycentric value Ga of the point spread function is in a portion such as “β” or “γ” shown in FIG. 7, the equation (3) can be applied.

  Next, consider how the pixel energy should be transferred from the original image data area to specific pixels in the restored image data area. Here, since the original image data Img is restored in the restored image data area, in order to obtain the energy “A” of a specific pixel of the original image data Img, the original image data area is changed to “A” in the restored image data area. There should be a transition to the corresponding specific pixel. Therefore, from equation (3), “A = k × A ′ / α” is shifted from the original image data area to a specific pixel corresponding to “A” in the restored image data area. Then, the pixel energy corresponding to “A” is removed from the original image data area. At this time, the change factor information data G is used to remove pixel energy from the original image data area so that there is no contradiction as a whole. For example, if the specific pixel corresponding to “A” is the pixel “S” shown in FIG. 7, the pixel energy to be removed from the pixel “S” in the original image data area is “k × A ′ / α × α”, The pixel energy to be removed from the pixel “S + 1” in the original image data area is “k × A ′ / α × β”, and the pixel energy to be removed from the pixel “S + 2” in the original image data area is “k × A ′ / α”. × γ ”. Then, the sum of these removed pixel energies becomes the pixel energy amount “A” transferred to the pixel “S” in the restored image data area. In this example, “k = 0.8” is set, and a specific state of pixel energy transition is described below.

The original image pixel energy “E” shown in step S101 of the first iteration process is shown in FIGS. In the pixel to be processed first, this original image pixel energy E is the energy E ₀ to be processed. On the other hand, in step S102, the reciprocal of the centroid value Ga of the point spread function is multiplied to obtain the transition pixel energy F _n . For example, when attention is paid to the pixel “S-3”, the centroid value Ga of the point spread function of the original image data (photographed data) (“E” in FIG. 9) is “0.5”. “0.8 × 60 / 0.5 = 96” is transferred from the original image data area to the pixel “S-3” in the restored image data area (step S103) (F ₁ (S-3) in FIG. 9). . Then, the pixel energy of “96” is removed from the original image data area. Then, the restored image data (R ₁ (S-3) in FIG. 9) becomes “96”, “0”. Here, since the data G of the change factor information is “0.5”, “0.3”, “0.2” as described above, “96 × from the pixel“ S-3 ”in the original image data area. 0.5 = 48 ”,“ 96 × 0.3 = 28.8 ”is removed from the pixel“ S-2 ”, and“ 96 × 0.2 = 19.2 ”is removed from the pixel“ S-1 ”. Should be (T ₁ (S-3) in FIG. 9). Therefore, the energy of the pixels “S-3” to “48”, the pixels “S-2” to “28.8”, and the pixels “S-1” to “19.2” in the original image data area is removed (step S104). ). At this time, the balance energy amount E _n for each pixel of the original image data area are each migration component is removed, a pixel "S-3" is "12", pixel "S-2" is "59.7", pixel "S-1" is "66.3", and the other pixels as the original is "89", "117", "105", "114", "142" (step S105) (in FIG. 9, _"E 1 ( S-3) ").

Next, when attention is paid to the pixel “S-2”, the energy “59.7” remains as the energy E _{0 to} be processed. When the same processing as the pixel “S-3” described above is performed, the pixel energy “95.52” is transferred from the original image data area to the pixel “S-2” in the restored image data area (F _{1 in} FIG. 9). (S-2)), the restored data R ₁ of the pixel “S-2” in the restored image data area is “95.52”, and the restored image data (R ₁ (S-2) in FIG. 9) is “96”. “95.52” “0”... Then, the pixels “S-2” to “47.76”, “28.656” from the pixel “S-1”, and “19.104” from the pixel “S” are removed from the original image data area. As a result, imaging data, i.e. the balance energy amount E _n for each pixel of the original image data area are each migration component is removed (in Fig. 9, "T _{1 (S-2)"),} the pixel "S-3" Is “12”, pixel “S-2” is “11.94”, pixel “S-1” is “37.644”, pixel “S” is “69.896”, and other pixels are the same as the original “117”, “105”, “114”, and “142” (“E ₁ (S-2)” in FIG. 9).

Next, paying attention to the pixel “S-1”, the same processing as the above-described pixel “S-3” is performed. Currently, energy of “37.644” remains as the energy E _{0 to} be processed. Then, the pixel energy of “60.23” is transferred from the original image data area to the pixel “S-1” in the restored image data area (F ₁ (S-1) in FIG. 9), and the pixels in the restored image data area The restored data R ₁ of “S-1” is “60.23”, and the restored image data (R ₁ (S-1) in FIG. 9) is “96”, “95.52”, “60.23”, “0”. "... Then, the pixels “S-1” to “30.115”, “18.069” from the pixel “S”, and “12.046” from the pixel “S + 1” are removed from the original image data area. As a result, imaging data, i.e. the balance energy amount E _n for each pixel of the original image data area are each migration component is removed (in Fig. 9, "T _{1 (S-1)"),} the pixel "S-3" Is “12”, pixel “S-2” is “11.94”, pixel “S-1” is “7.529”, pixel “S” is “51.825”, and pixel “S + 1” is “104. 954 ”, and the other pixels are“ 105 ”,“ 114 ”, and“ 142 ”as originally (“ E ₁ (S-1) ”in FIG. 9).

In this way, the pixel energy is sequentially shifted for all the pixels. In the first process (n = 1), the original image pixel energy E of all the pixels “S-3”, “S-2”, “S-1”... “S + 4” is not moved to the restored image data area. balance energy amount E _n for each pixel of the original image data area is left as a larger value. The remaining energy maximum value “E _max ” (“19.02” in the pixel “S + 4”) is stored in the memory of the control unit 4 (step S106). Then, it is determined whether or not the remaining energy amount maximum value E _max is less than a predetermined value X (for example, “5” in this example) (step S107). As a result of the above processing, since E _max > X, the same pixel energy transfer processing is performed as the second time (n = 2). And then (until the balance energy amount E _n for all of the pixels closer to "0") until satisfying E _{max> X,} repeats the same migration process.

Here, a particular pixel of FIG. 9, a result of the migration the migration pixel energy F _n, which may or restored data R _n is I exceeds a predetermined upper limit value, is the balance energy amount E _n becomes a negative value . The occurrence of this state means that the transition pixel energy F _n has been set to an inappropriate value. Therefore, a method for reviewing the transition value without any contradiction as a whole will be described below even in such a case.

(Transition value review method 1)
For pixels balance energy amount is negative does not migrate the pixel energy F _n should be moved to the restored image data area, as it uses the value of the previous processing, to subject the next migration process, when the We expect that the pixel energy to be removed will not be an inappropriate value. Next time, the remaining energy amount of the surrounding pixels decreases, and the pixel energy value to be removed from the pixel that becomes negative when the current transition is made, that is, the pixel energy value to be extracted becomes smaller than this time. There is a high possibility of being “0” or more. However, for example, when the previous value is used as it is when the pixel “S” becomes negative, the remaining energy amount of the processed pixels such as the pixels “S-2” and “S-1” approaches “0”. If it is, it will not be restored indefinitely and will not converge. Therefore, in this case, it is desirable to review the restoration values of the pixels “S-2” and “S-1”.

(Transition value review method 2)
The concept of the review method after migration will be described based on the conceptual diagram shown in FIG. Already correction amount a portion of the migrated pixel energy restored image data area (hereinafter, referred to as corrected energy cE) back to the original image data area as the remainder energy amount E _n to 0 or more. At this time, the correction energy cE is superimposed and integrated with the data G of the change factor information and returned to the original image data area. Then, the transition value can be reviewed without any contradiction as a whole.

Next, the remainder amount of energy E _n became negative value will be described the process of reviewing the transition value to "0" based on the idea described above. Specifically, the pixel energy transfer process of FIG. 9 is performed in the order of the pixels “S-3”, “S-2”... “S + 4” in accordance with the process routine of FIG. when in Figure 3 of the pixel energy transition process to a pixel "S-3" in the restored data region when) of n = 3, the balance energy amount E _n of a pixel "S-1" of the original image data area, "- 0.624 ". For reference, FIG. 11 shows the state of processing when n = 2 in FIG. 3, and FIG. 12 shows the state of processing when n = 3 in FIG. 3 (only three digits after the decimal point are displayed. The same applies to the description of FIGS. 14 and 15.) A process for setting the value of “−0.624” to “0” will be described. In this process, the transition value (n to the pixel “S-3” in the restored image data area, which affects the pixel “S-1” in the original image data area, depending on how the pixel energy is distributed as shown in FIG. = 3) The following processing is performed on the assumption that it is inappropriate. Hereinafter, the process will be described with reference to the table of FIG.

In order to set the remaining energy amount E _n (= −0.624) of the pixel “S-1” in the original image data area to “0”, the restored data R _n of the pixel “S-3” in the restored image data area From (= 119.040), the correction energy cE is returned to the original image data area. Here, the ratio at which the pixel energy of the pixel “S-3” in the restored image data area is dispersed to the pixel “S-1” in the original image data area is “γ = 0.2 from the energy dispersion ratio of FIG. It is. Therefore, the correction energy cE becomes cE = −E _n /γ=−(−0.624)/0.2=3.120 from cE × γ = −E _n .

The correction energy cE (= 3.120) to the pixel “S-3” in the restored image data area is the same as that in the pixels “S-1”, “S-2”, and “S-3” in the original image data area. From the energy dispersion ratio shown in FIG. 7, it can be considered that the pixel energy is dispersed as follows.
(1) Pixel “S-3”: Since the dispersion ratio is “0.5”, 3.120 × 0.5 = 1.560
(2) Pixel “S-2”: Since the dispersion ratio is “0.3”, 3.120 × 0.3 = 0.936
(3) Pixel “S-1”: Since the dispersion ratio is “0.2”, 3.120 × 0.2 = 0.624

  The correction value cE (= 3.120) is removed from the pixel “S-3” in the restored image area, and the correction energy is returned to the original image area in accordance with the energy dispersion ratio shown in FIG. Ends. In the transition value review method 2, the transition value (n = 3 times) to the pixel “S-3” in the restored image data area, which affects the pixel “S-1” in the original image data area, is inappropriate. The processing is performed on the premise that there is, but the transition value (n = 3 times) to the pixel “S-2” in the restored image data area, which affects the pixel “S-1” in the original image data area, is invalid. Based on the assumption that it is appropriate, processing can be performed based on the same concept.

(Transition value review method 3)
For even transition value reviewing method 3, similarly reviewing method 2 of the transition value, the balance energy amount E _n of a pixel of the original image data area "S-1" when n = 3, in Figure 3, in the table of FIG. 14 As shown, “−0.624” is changed to “0”. For that purpose, the pixel energy distribution shown in FIG. 7 affects the pixels “S-3” and “S-2” in the restored image data area, which affects the pixel “S-1” in the original image data area. Process on the assumption that the migration value is inappropriate. Hereinafter, the process will be described with reference to the table of FIG.

In order to set the remaining energy amount E _n (= −0.624) of the pixel “S-1” in the original image data area to “0”, the restored data R _n of the pixel “S-3” in the restored image data area From (= 119.040), the correction energy cE _S-3 derived from the following equation (4) is returned to the original image data area. Here, “cE _S-3 ” is the correction energy of the restoration data R _n of the pixel “S-3” in the restoration image data area, and is the total energy returned to the original image data area. Further, the correction energy cE _S derived from the following equation (5) is returned to the original image data area from the restoration data R _n (= 105.408) of the pixel “S-2”. Here, “cE _S-2 ” is the correction energy of the restoration data R _n of the pixel “S-2” in the restoration image data area, and is the total energy returned to the original image data area.

cE _S-3 = −E _n × P / (P × γ + Q × β) (4)
_{cE S-2 = -E n ×} Q / (P × γ + Q × β) ...... (5)
Here, cE _S-3 : cE _S-2 = P: Q. “Β” is a ratio (FIG. 7) in which the pixel energy of the pixel “S-3” in the restored image data area is dispersed to the pixel “S-2” in the original image data area, and “γ” is the restored image. This is a ratio (FIG. 7) in which the pixel energy of the pixel “S-3” in the data area is distributed to the pixel “S-1” in the original image data area.

In this example, the process proceeds with “P: Q = 1: 1”. The ratio of P and Q can be set arbitrarily. For example, P: Q = γ: β may be set corresponding to the dispersion ratio of the original image data area to the pixel “S−1”. In this example, since “β = 0.3” and “γ = 0.2”, the equations (4) and (5) are as follows.
cE _S-3 = − (− 0.624) × 1 / (1 × 0.2 + 1 × 0.3) = 1.248
cE _S-2 = − (− 0.624) × 1 / (1 × 0.2 + 1 × 0.3) = 1.248

Then, the correction energies cE _S-3 and cE _S-2 (= 1.248) of the pixels “S-3” and “S-2” in the restored image data area are the pixels “S” in the original image data area before the transfer process. In “S-3”, “S-2”, “S-1”, and “S”, it can be considered that the pixel energy is dispersed as follows according to the dispersion method of the pixel energy shown in FIG.
(1) Since the dispersion ratio of the pixel “S-3”: cE _S-3 is “0.5”, 1.248 × 0.5 = 0.624
(2) Pixel “S-2”: Since the dispersion ratio of cE _S-3 is “0.3” and the dispersion ratio of cE _S-2 is “0.5”, 1.248 × 0.3 + 1.248 × 0.5 = 0.998
(3) Since the dispersion ratio of the pixel “S-1”: cES _-3 is “0.2” and the dispersion ratio of cES _-2 is “0.3”, 1.248 × 0.2 + 1248 × 0.3 = 0.624
(4) Since the dispersion ratio of the pixel “S”: cE _S−2 is “0.2”, 1.248 × 0.2 = 0.250
Then, the correction energies cE _S-3 and cE _S-2 are respectively applied to the pixels “S-3”, “S-2”, “S-1”, and “S” in the original image data area, and the “return amount” in FIG. By returning the value as the sum of “1” and “return amount 2”, the transition value review process is completed.

(Transition value review method 4)
Review method 4 transition values, as well reviewed methods 2 and 3 of the transition value, the balance energy amount E _n of a pixel "S-1" of the original image data region when the n = 3 in FIG. 3, the table of FIG. 15 As shown in the figure, “−0.624” is set to “0” or more. Therefore, the transition value “1.156” to the pixel “S-1” of the restored image data area of the previous time (n = 2 times in FIG. 3) affecting the pixel “S-1” of the original image data area. ”Is processed in the same manner as the above-described review method 2 and 3 of the transition value. Hereinafter, the process will be described with reference to the table of FIG.

In order to set the remaining energy amount E _n (= −0.624) of the pixel “S-1” in the original image data area to “0”, the restored data R _n of the pixel “S-1” in the restored image data area The correction energy cE is returned to the original image data area. Here, the ratio that the pixel energy of the pixel “S-1” in the restored image data area is dispersed to the pixel “S-1” in the original image data area is “α = 0.5 from the energy dispersion ratio of FIG. It is. Therefore, the correction energy cE becomes cE = −E _n /γ=−(−0.624)/0.5=1.248 from cE × γ = −E _n . Then, in the pixels “S−1”, “S”, and “S + 1” in the original image data area, it can be considered that the pixel energy is dispersed as follows according to the dispersion method of the pixel energy shown in FIG.
(1) Pixel “S-1”: Since the dispersion ratio is “0.5”, 1.248 × 0.5 = 0.624
(2) Pixel “S”: Since the dispersion ratio is “0.3”, 1.248 × 0.3 = 0.374
(3) Pixel “S + 1”: Since the dispersion ratio is “0.2”, 1.248 × 0.2 = 0.250
Then, the correction energy cE (= 1.248) is removed from the pixel “S-1” in the restored image area, and the correction energy is returned to the original image area according to the energy dispersion ratio shown in FIG. The review process ends. As the advantages of the revision process 4 are inadequate transition value to the rest energy amount E _n is the same as pixel becomes less than 0 pixels of the original image data area, to be able to review the process of transition values It is.

(How to review other transition values)
The migration values of all pixels energy transfer process the balance energy amount E _n is the effect of less than 0 in the original image data area, reviewing accordance idea of reviewing the review methods 2, 3, and 4 of the transition value of the above You can also. Further, in reviewing methods 2, 3 and 4 of the transition value of the above, rather than the process of the the balance energy amount E _n of the original image data area is less than 0 to "0", a value exceeding "0" It can also be a process to do. For example, when it is considered that noise is included in the original image data area, it is preferable to perform a process of setting a value exceeding “0” (a value obtained by adding noise to “0”). Furthermore, in reviewing methods 2, 3, and 4 of the transition value of the above, that the balance energy amount E _n of a pixel of the original image data region when the n = 3 in FIG. 3 is a negative value, the previous (n = 3) The transition value can be reviewed on the assumption that the transition value to the pixel of the restored image data area in the previous round is inappropriate.

The transition value at the time when the “return process” in the transition value review method 1 described above or the “return process” in the transition value review method 2, 3, or 4 described above is executed during the process shown in FIG. FIG. 16 shows a processing flowchart for explaining a processing routine related to the review method. After step S105 in the processing flow of FIG. 3, balance energy amount E _n performs negative value determines whether (step S201). _If En is not a negative value, the processing after step S106 in FIG. 3 is performed (step S202). If E _n is a negative value, perform energy transfer value reviewing processing review methods 1, 2, 3 or 4, the transition value of the above (step S203), then the process proceeds to step S202.

In the signal processing apparatus 1 according to the present embodiment described above, upon processing, in step S104, S107, and pre-processing frequency, whether any of the determination reference value balance energy E _n have approximated to "0" One or both can be set. For example, an arbitrary number such as 20 times or 50 times can be set as the number of times of processing. In addition, the balance energy E _n to stop the process is set to "5" of the 8-bit (0 to 255) in the value of the approximate value of whether've been approximated to "0", to terminate the processing Once you become 5 or less Or “0.5” can be set and the processing can be terminated when “0.5” or less. This set value can be set arbitrarily. When both the number of processing times and the judgment reference value are input, it is preferable that the processing is stopped when either one is satisfied. When both settings are possible, the determination reference value may be given priority, and if the predetermined number of processes does not fall within the determination reference value, the predetermined number of processes may be repeated.

  In the description of the present embodiment, the information stored in the recording unit of the processing unit 4 is used without using the information stored in the factor information storage unit 7, but the known information stored therein is used. It is also possible to use data such as deterioration factors such as optical aberration and lens distortion. In this case, for example, in the processing method (repetitive processing) of the previous example (FIG. 3), it is preferable to perform processing by combining blur information and optical aberration information as one deterioration factor. You may make it perform the decompression | restoration process by the information of optical aberration, after complete | finishing the process by information. Further, the factor information storage unit 7 may not be installed, and the image may be corrected or restored only by dynamic factors at the time of shooting recorded in the processing unit 4, for example, only blurring.

  Further, when the number of repetitions of processing is set automatically or fixedly on the signal processing device 1 side, the set number of times may be changed by the data G of the change factor information. For example, when the data of a certain pixel is distributed over many pixels due to blurring, the number of repetitions may be increased, and when the dispersion is small, the number of repetitions may be decreased.

Moreover, when generating the restored data R _n to be output image, depending on the data G of fluctuation-factor information, in some cases data that would leave the region outside of the image to try to restore occurs. In such a case, data that protrudes outside the area is input to the opposite side. If there is data that should come from outside the area, the data may be brought from the opposite side. For example, when data allocated to a lower pixel is generated from the data of the pixel XN1 positioned at the bottom in the area, the position is outside the area. Therefore, the data is processed to be allocated to the pixel X11 located at the top right above the pixel XN1. Similarly, the pixel N2 adjacent to the pixel XN1 is assigned to the topmost pixel X12 (= next to the pixel X11) directly above.

  The restoration algorithm of this embodiment has an advantage that the data area of the signal processing apparatus 1 can be reduced. The reason is that only the original image data area and the restored image data area are necessary for the restoration process. Further, in the restoration process, the restoration algorithm of this embodiment only repeats the movement of the pixel energy using the data G of the known change factor information, so that a rapid process is possible. Further, the data area is not permanent, and processing may be performed after setting a temporary data area.

The signal processing apparatus 1 in this embodiment has been described above, but various modifications can be made without departing from the gist of the present invention. For example, the processing performed by the processing unit 4 is configured by software, but may be configured by hardware composed of parts that perform part of the processing. Further, the change factor information data G includes not only the deterioration factor information data but also information for simply changing an image and information for improving an image contrary to deterioration. Furthermore, during the repetition of the process, without comparison between the maximum value E _max and the predetermined value of the value of the remainder energy amount E _n for each pixel each have (X), the average value or the sum of the balance energy amount E _n Comparison with values can be made. By doing so, the processing speed is improved. It is also possible to carry out the maximum value E _max values of balance energy amount E _n for each pixel each have _a plurality of average or total value is compared with the plurality of predetermined values corresponding to each of these values .

  Further, when the number of repetitive processes is set automatically or fixedly on the signal processing apparatus 1 side, the set number of times may be changed by the data G of the change factor information. For example, when the data of a certain pixel is distributed over a large number of pixels due to blurring, the number of repetitions may be increased, and when the distribution is small, the number of repetitions may be decreased.

  In the above-described embodiment, the restoration target is image data. However, the concept and method of the restoration process and the review method can be applied to any data restoration process. For example, application to restoration of audio data, restoration of seismic wave data, and the like is possible. Further, in the above-described embodiment, an example in which image data is blurred at each location is shown. However, these ideas can be applied to an image that varies depending on the pixel position and non-linear images such as gamma correction. And methods and review methods can be applied.

  Each processing method and review method described above may be programmed. Alternatively, the program may be stored in a storage medium, such as a CD (Compact Disc) DVD or a USB (Universal Serial Bus) memory, and read by a computer. In this case, the signal processing apparatus 1 has means for reading a program in the storage medium. Further, the program may be stored in an external server of the signal processing apparatus 1, downloaded as necessary, and used. In this case, the signal processing apparatus 1 has communication means for downloading the program in the storage medium.

Claims (12)

From the original signal data (hereinafter referred to as change data) in which a change such as deterioration has occurred, the signal before the change, the signal that should have been originally acquired, or the data of the approximate signal thereof (hereinafter referred to as the original signal data). In a signal processing apparatus having a processing unit for restoring)
A change data area in which the change data is stored and a restoration data area in which data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process are provided, and the processing unit includes: Using the change factor information data that causes the change, the energy of the change data is transferred from the change data area to the restored data area, the restored data is generated, and the change data remaining after the migration is generated. If the remaining data in the area is replaced with the change data and the same process is repeated, and the energy value of the remaining data becomes less than zero in the process of the repeated process, the area is already transferred to the restored data area. The change data is used so that the energy value of the remaining data becomes zero or more by using the change factor information for a part of the energy. While process for returning to the band allowed to proceed the repetitive processing, the signal processing device the data to be formed on the restored data area when the repetitive processing ends, characterized in that the above original signal data.   2. The signal processing device according to claim 1, wherein when the energy value of the remaining data is less than zero, processing is performed so that the value becomes zero or more.   At the time of the iterative process, each time the process is repeated, the energy that shifts to the restored data area is added to the restored data that is already stored in the restored data area, and the energy value of the remaining data is in a range of zero or more. The signal processing apparatus according to claim 1, wherein the signal processing unit performs processing to approach zero. In a signal processing apparatus having a processing unit for restoring original signal data consisting of a plurality of elements from change data consisting of a plurality of elements,
A change data area in which the change data is stored and a restoration data area in which data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process are provided, and the processing unit includes: The element energy in one element of the change data is transferred from the change data area to the restored data area by using the centroid value of the response characteristic function included in the change factor information data that causes the change. The element energy corresponding to the element energy is removed from the change data area using the change factor information data, and the process for the one element is sequentially performed for the other elements, and the restored data Generate the restoration data in the area and replace the remaining data of the change data area remaining by the exclusion with the change data. The process is repeated for each element, and the element energy that is transferred to the restored data area is added to the restored data each time the process is repeated, and a process for generating new restored data is performed. When any element energy value of the remaining data is less than zero, a part of the element energy that has already been transferred to the restored data area is converted to an element energy that is less than zero using the change factor information. The above-mentioned series of processing is performed while performing processing to return to the change data region so that the value becomes zero or more, processing to bring the remaining data close to zero in a range of zero or more, and the restored data region at the end of processing A signal processing apparatus characterized in that the restored data formed in the step is used as the original signal data.   The processing unit, when generating the restoration data, performs a process of stopping when an energy value of the remaining data is equal to or less than a predetermined value within a range of zero or more or smaller than the predetermined value. 5. The signal processing device according to any one of 1 to 4.   5. The signal according to claim 1, wherein when generating the restoration data, the processing unit performs a process of stopping when the restoration data is generated a predetermined number of times. 6. Processing equipment.   The processing unit, when generating the restoration data, when the energy value of the remaining data when the number of times the restoration data is generated reaches a predetermined number is less than or equal to a predetermined value within a range of zero or more or smaller than the predetermined value 5. The signal processing apparatus according to claim 1, wherein the signal processing apparatus further performs a process that is repeated a predetermined number of times when the value exceeds the predetermined value or exceeds the predetermined value. In a signal processing apparatus having a processing unit for restoring original signal data consisting of a plurality of elements from change data consisting of a plurality of elements,
A change data area in which the change data is stored and a restoration data area in which data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process are provided, and the processing unit includes: The energy in one element of the change data is transferred from the change data area to the restored data area by using change factor information data that causes change, and energy corresponding to the transferred energy is changed. The process for excluding the change factor information data from the change data area is performed, and the process for the one element is sequentially performed for all the other elements to generate the restored data in the restored data area. If the value of the remaining data in the change data area remaining after the transition is less than or equal to a predetermined value in the range of zero or more or smaller than the predetermined value, If the value of the remaining data is greater than the predetermined value or greater than the predetermined value, the remaining data is replaced with the change data and the same processing is performed. Each time, each element energy value of the remaining data is less than zero by adding the element energy that moves to the restoration data area to the restoration data, and generating new restoration data. In this case, a part of the element energy that has already been transferred to the restoration data area is converted to the change data so that the element energy value that is less than zero of the remaining data becomes zero or more by using the change factor information. A signal processing apparatus that compares the remaining data amount with the predetermined value while performing processing to return to an area.   9. The signal processing apparatus according to claim 8, wherein the processing unit performs a process of stopping when the restoration data is generated when the number of times the restoration data is generated reaches a predetermined number.   When generating the restored data, the processing unit compares one or more of the maximum value, the average value, or the total value of the remaining data of each element with the predetermined value. The signal processing apparatus according to claim 8.   The restoration process is performed when the restoration data is generated once or a plurality of times, and the element that has moved to the restoration data area before the time when any one of the element energy values of the remaining data becomes less than zero. The signal processing apparatus according to claim 4, wherein the signal processing apparatus performs energy.   12. The signal processing apparatus according to claim 1, wherein the signal data is image data.

Images