WO2007122883A1 - Signal processing apparatus - Google Patents

Abstract

A signal processing apparatus wherein a changed signal, such as a degraded signal or the like, can be restored with realizability achieved without increasing the apparatus scale. The signal processing apparatus (1) has a processing part (4) that restores a source signal data, which is, for example, a signal before being changed, from an original signal data in which the change, such as degradation, has occurred (and which will be referred to as “changed data” hereinafter). There are provided a changed data area in which the changed data is stored and a restored data area in which the signal data as restored is stored each time the restoring process is done. The processing part performs a repetitive processing for obtaining the source signal. When the energy value of the residual data is less than zero during the repetitive processing, some of the energy having already moved to the restored data area is placed, by use of change factor information, back to the changed data area such that the energy value of the residual data becomes equal to or greater than zero. In this way, the repetitive processing is performed such that the data formed in the restored data area after completion of the repetitive processing will be used as the source signal data.

Description

 Specification

 Signal processing device

 Technical field

 [0001] The present invention relates to a signal processing device.

 Background art

 Conventionally, it has been known that when an image is processed by an image processing apparatus such as a camera, the image sometimes deteriorates. Causes of image degradation include camera shake during shooting, various aberrations of the optical system, and lens distortion.

 [0003] To correct camera shake during shooting, a method of moving a lens and a method of circuit processing are known. For example, as a method for moving a lens, a method is known in which camera shake is detected and correction is performed by moving a predetermined lens in accordance with the detected camera shake (see Patent Document 1). In addition, as a circuit processing method, a change in the optical axis of the camera is detected by an angular acceleration sensor, and a transfer function representing a blurring state at the time of photographing the detected angular velocity is taken. It is known that an image is restored by performing inverse transformation of a function (see Patent Document 2).

 [0004] In addition to general captured images, various signals such as sound, X-ray photographs, microscopic images, and seismic waveforms are known to deteriorate or change due to blurring or other causes. .

 [0005] Patent Document 1: Japanese Patent Laid-Open No. 6-317824 (see abstract)

 Patent Document 2: JP-A-11 24122 (see abstract)

 Disclosure of the invention

 Problems to be solved by the invention

[0006] The camera adopting the camera shake correction described in Patent Document 1 requires a hardware space for driving a lens, such as a motor, and becomes large. In addition, such hardware itself and a drive circuit for moving the hardware are required, which increases costs. In addition, in the case of camera shake correction described in Patent Document 2, the above-described problems are eliminated, but the following problems are present. In other words, image restoration is performed by inverse transformation of the acquired transfer function. Theoretically holds true, but as a practical matter, image restoration is difficult for the following two reasons:

[0007] First, the value of the transfer function to be obtained fluctuates greatly due to these slight fluctuations that are very vulnerable to noise and blur information errors. For this reason, the corrected image obtained by the inverse transformation has no camera shake and is far from the image shot in the state, and cannot be used in practice. Second, when performing inverse transformation considering noise, etc., a method of estimating the solution by singular value decomposition of the solution of simultaneous equations can be adopted, but the calculated value for the estimation becomes astronomical. In practice, there is a high risk of being unable to solve.

 [0008] The above-mentioned problems that occur in images also appear in various general data, and signal restoration by inverse transformation of the transfer function is not only accurate when the acquired transfer function is inaccurate, but also accurate. Even so, it is difficult. However, it is impossible to obtain a transfer function that is 100% accurate when the natural world is targeted.

 [0009] As described above, an object of the present invention is to provide a signal processing device that prevents a device from becoming large and restores a signal and has a realistic circuit processing method.

 Means for solving the problem

[0010] In order to solve the above problems, the signal processing apparatus of the present invention should acquire the signal before the change or the original signal data from the original signal data (hereinafter referred to as change data) in which a change such as deterioration has occurred. A processing unit that restores the data of the signal or the approximate signal thereof (hereinafter referred to as the original signal data! /, Etc.), the change data area in which the change data is stored, and each restoration process A restoration data area for storing the restored signal data (hereinafter referred to as restoration data) is provided, and the processing unit uses the energy of the change data and the change factor information data that causes the change. The transition from the changed data area to the restored data area is generated, the restored data is generated, and the remaining data of the changed data area remaining by the migration is replaced with the changed data, and the same process is repeated. When the energy value of the remaining data becomes less than zero in the process of repeated processing, a part of the energy that has already been transferred to the restored data area is converted to the remaining data using the change factor information. Repeat the process while returning to the change data area so that the energy value becomes zero or more. The data formed in the restored data area at the end of repeated processing is used as the original signal data.

 [0011] According to the present invention, since the original signal data is assumed to have changed to change data according to the change factor information data, the energy of the change data can be obtained by using the change factor information data that forms the same filter. Is transferred to the restoration data area, the original signal data is reliably restored as restoration data in the restoration data area. Also. As a result of the energy shift, it is possible to avoid a situation that could not occur theoretically, that is, the energy value of the remaining data is less than zero, so the restoration accuracy of the restoration data can be improved. Here, the change data stored in the change data area may be obtained by processing the change data while keeping the energy state of the change data as it is (the same applies hereinafter). ). In addition, the restoration data stored in the restoration data area may be the restoration data that has been processed after the state of energy constituting the restoration data is left as it is (the same applies hereinafter). Furthermore, “migration” literally moves the energy value from the changed data area to the restored data area, and removes the energy from the changed data area power and generates new energy in the restored data area. (Including the same). Further, the change data area and the restored data area include both temporarily formed areas and permanently formed areas (the same applies hereinafter).

 In another invention, in addition to the above-described invention, when the energy value of the remaining data is less than zero, the value is processed to be zero or more. If the energy value of the remaining data becomes zero, it can be considered that the restoration accuracy of the restoration data restored from the change data is very good.

 [0013] In addition to the above-described invention, another invention performs a process of adding energy transferred to the restoration data area to the restoration data already stored in the restoration data area at the time of repetition processing, and the remaining data The energy value of is close to zero in the range of zero or more. If the remaining data approaches zero in the range of zero or more, most of the energy in the changed data area shifts to the restored data area, so that the restored data approaches the original signal data.

[0014] In order to solve the above-described problem, another signal processing apparatus of the present invention includes a plurality of elements. The processing unit that restores the original signal data, which has multiple elemental forces, from the digitized data, the change data area in which the change data is stored, and the data of the restored signal for each restoration process (hereinafter, , The restoration data area for storing the data, and the processing unit uses the element energy of one element of the change data as the center of gravity value of the response characteristic function of the change factor information data that causes the change. Is used to move to the change data area power restoration data area, and the element energy corresponding to the transferred element energy is also excluded using the change data area data. The process for one element is also performed sequentially for the other elements, and restored data is generated in the restored data area, and the remaining data in the changed data area remaining after the exclusion is changed to the change data. The same process is repeated for each element, and the element energy that moves to the restoration data area is added to the restoration data each time it is repeated. If any element energy value of the remaining data becomes less than zero in the process of the above, a part of the element energy that has already moved to the restored data area is less than zero using the change factor information. The process returns to the change data area so that the element energy value becomes zero or more, while proceeding with a series of processes, the process to bring the remaining data closer to zero within the range of zero or more, and at the end of the process The restored data formed in the restored data area is used as the original signal data!

According to the present invention, since it is assumed that the data of the original signal has changed to change data according to the change factor information data, the centroid value of the response characteristic function (energy is the energy) By using the reciprocal of the most concentrated value), most of the energy of the change data is transferred to the restoration data area, and the element energy corresponding to the transferred energy is also changed in the change data area power. If the energy value of the remaining data (hereinafter referred to as the remaining energy amount) can be made close to zero, the original signal data is reliably restored as restored data in the restored data area. Furthermore, as a result of energy transfer, it is possible to avoid a theoretically impossible situation where the energy value of any of the elements constituting the remaining data is less than zero, thus improving the restoration accuracy of the restoration data. be able to. Response characteristic functions include impulse response functions and unit response functions. And signal data Is an image data, the impulse response function is a point spread function.

 [0016] A signal processing device according to another invention is based on the above-described invention, and the processing unit, when generating the restoration data, uses a predetermined value less than or equal to a predetermined value within a range of the energy value power of zero or more of the remaining data. When it gets smaller, it is stopped. When this configuration is adopted, the processing is stopped even if the remaining energy amount does not become “0”, so that it is possible to prevent a long processing time. In addition, since the value is less than the predetermined value, the restored data to be approximated is closer to the original signal data before the change (before deterioration, etc.) that is the source of the change data. Furthermore, if there is a noise, etc., the force that tends to cause a situation in which the remaining energy amount cannot be “0” in reality. Even in such a case, the process is repeated indefinitely. It will not be.

 [0017] In addition to the above-described invention, in the signal processing device according to another invention, the processing unit performs a process of stopping when the number of times of generating the restoration data reaches a predetermined number when generating the restoration data. . When this configuration is adopted, the processing is stopped regardless of whether the remaining energy amount becomes “0”, so that the processing can be prevented from being prolonged. In addition, since the processing is continued until a predetermined number of times, the approximate restoration data is closer to the original signal data before the deterioration that is the source of the change data. In addition, when there is noise, the power that tends to cause a situation where the remaining energy amount does not become “0” in reality. Will not be repeated.

 [0018] A signal processing device according to another invention is based on the above-described invention, and the processing unit, when generating the restored data, checks the remaining data when the number of times the restored data is generated reaches a predetermined number. If the energy value is less than or equal to a predetermined value in the range of zero or more, or is smaller than the predetermined value, the process is stopped. In the present invention, since the number of times of processing and the remaining energy amount are combined, the restoration accuracy is improved compared to the case where the number of processing times is simply limited or the remaining energy amount is limited. Processing that balances the shortness of processing time can be achieved.

[0019] In order to solve the above-described problem, another signal processing apparatus of the present invention includes a processing unit that restores original signal data having a plurality of elemental forces from change data including a plurality of elements, and A change data area in which data is stored and a restoration data area in which data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process are provided. The energy in one element is transferred from the change data area to the restoration data area using the change factor information data that causes the change, and the energy corresponding to the transferred energy is changed in the change data area. Perform the exclusion process using the factor information data, perform the process for one element for all other elements in sequence, generate the restored data in the restored data area, and change data area remaining by the migration If the remaining data value is less than or equal to a predetermined value within the range of zero or more, the processing is stopped and the restored data is treated as the original signal data. If the value is greater than the specified value or greater than or equal to the specified value, the remaining data is replaced with the change data and the same process is repeated, and the element energy that moves to the restored data area is added to the restored data each time it is repeated. If any element energy value of the remaining data is less than zero when processing to generate new restoration data is performed, a part of the element energy that has already moved to the restoration data area is displayed as the change factor information. The remaining data amount is compared with a predetermined value while performing a process of returning to the change data area so that the element energy value that becomes less than zero of the remaining data becomes zero or more.

According to the present invention, since it is assumed that the data of the original signal has changed to change data according to the change factor information data, the energy of the change data can be obtained by using the change factor information data that is the same filter. Can be restored to the restored data area, the original signal data is reliably restored in the restored data area. Furthermore, as a result of energy transfer, it is possible to avoid a situation that could not occur theoretically, in which the energy value of any element constituting the remaining data is less than zero. Can do. Also, change energy information is used to transfer energy from the change data area to the restoration data area, and only when the remaining energy amount in each element of the change data area exceeds the specified value or exceeds the specified value. Since the remaining data is replaced with change data and the same process is repeated, the restoration process can be performed quickly. Furthermore, since restoration processing is performed by transferring energy from the change data area to the restoration data area, the apparatus does not increase in size with little hardware increase. Because of this, signal restoration In this case, a signal processing apparatus having a realistic circuit processing method can be obtained.

 In the signal processing device according to another invention, in addition to the above-described invention, the processing unit performs a process of stopping when the number of times of generating the restored data reaches a predetermined number when generating the restored data. . When this configuration is adopted, the processing is stopped regardless of whether the remaining energy amount becomes “0”, so that the processing can be prevented from being prolonged. Further, since the processing is continued up to a predetermined number of times, the restored data is closer to the original signal data. Furthermore, when there is noise or the like, a situation where the remaining energy amount does not become “0” is likely to occur in reality, but in such a case, the processing is repeated indefinitely. Such a problem does not occur when the composition is adopted.

 [0022] In addition to the above-described invention, the signal processing device according to another invention may be one of the maximum value, the average value, or the total value of the remaining data values of each element when the restoration data is generated or Comparison with a predetermined value is performed for a plurality. When this configuration is adopted, the remaining energy amount in each element constituting the change data can be brought close to zero, so that the degree of approximation between the restored data and the original signal data can be further increased.

 [0023] In the signal processing device according to another invention, in addition to the above-described invention, the returning process is performed when the element energy value of any one of the remaining data is zero when the restoration data is generated once or a plurality of times. The target energy is the element energy that has moved to the restored data area before that time. When this configuration is adopted, it is possible to avoid a situation that could not occur theoretically, where the energy value in any element of the remaining data becomes less than zero as a result of energy transfer. Accuracy can be improved.

 [0024] In addition to the above-described invention, a signal processing apparatus according to another invention uses signal data as image data. As a result, even if image degradation occurs due to camera shake, the original image that has undergone degradation, the image before the change, the image that should have been taken, or an approximate image thereof (hereinafter referred to as the original image). Can be restored.

 The invention's effect

 [0025] According to the present invention, in restoring a signal that has changed due to deterioration or the like, it is possible to prevent an increase in the size of the apparatus and to provide a realistic signal processing apparatus.

Brief Description of Drawings FIG. 1 is a block diagram showing a main configuration of a signal processing device according to an embodiment of the present invention. 圆 2] An external perspective view showing an outline of the signal processing device shown in FIG. It is a figure for demonstrating.

 3 is a processing flow diagram for explaining a processing routine related to an image restoration processing method (repetitive processing) performed by a processing unit of the signal processing device shown in FIG. 1.

[4] FIG. 4 is a diagram for explaining the concept of the processing method performed by the processing unit of the signal processing device shown in FIG.

 FIG. 5 is a diagram for specifically explaining the processing method shown in FIG. 3 using camera shake as an example, and is a table showing the concentration of pixel energy when there is no camera shake.

 FIG. 6 is a diagram for specifically explaining the processing method shown in FIG. 3 using camera shake as an example, and is a diagram showing image data when there is no camera shake.

 FIG. 7 is a diagram for specifically explaining the processing method shown in FIG. 3 with an example of camera shake, and a diagram showing dispersion of pixel energy when camera shake occurs.

 FIG. 8 is a diagram for explaining an example of a camera shake situation when the pixel energy dispersion shown in FIG. 7 occurs.

 FIG. 9 is a diagram illustrating the processing method shown in FIG. 3 using the image data shown in FIG. 8 degraded by camera shake as an example.

 FIG. 10 is a conceptual diagram showing a concept of a method for reviewing transition values in transition processing when the processing method shown in FIG. 3 is executed.

 FIG. 11 is a diagram showing a state of processing of the image data shown in FIG. 8 by the processing method in FIG. 3, and a state of processing at the stage of “n = 2”.

 FIG. 12 is a diagram showing a state of processing of the image data shown in FIG. 8 by the processing method in FIG. 3, and a state of processing at the stage of “n = 3”.

 FIG. 13 is a diagram for explaining an example of a transition value review method 2 in the transition process when the process shown in FIG. 3 is executed.

FIG. 14 is a diagram for explaining an example of a transition value review method 3 in the transition process when the process shown in FIG. 3 is executed. FIG. 15 is a diagram for explaining an example of a transition value review method 4 in the transition process when the process shown in FIG. 3 is executed.

 FIG. 16 is a process flow diagram for explaining a processing routine according to a method for reviewing transition values when the processing method shown in FIG. 3 is executed.

 Explanation of symbols

[0027] 1 Signal processing device

 2 Receiver (shooting unit)

 3 Control system

 4 Processing section

 5 Recording section

 6 Detector

 7 Factor information storage

 G Change factor information data (degradation factor information data)

 Centroid value of Ga point spread function

 Img 'Original image data (captured image)

 Img No deterioration! Original correctness! / Image data (original image)

 E Original image pixel energy

 F transition pixel energy

 R Restored data

 cE ne if positive energy

 E Remaining energy (energy to be processed)

 X Predetermined value of remaining energy

 τ Pixel energy removed from the original image data area

 cE correction energy

 BEST MODE FOR CARRYING OUT THE INVENTION

[0028] Hereinafter, a signal processing device 1 according to an embodiment of the present invention will be described with reference to the drawings. The signal processing device 1 is an image processing device, which is used as a consumer camera, such as a force monitoring camera, a television camera, and a handy video camera. It can also be applied to devices other than cameras, such as a camera for other uses such as a camera, an endoscopic camera, a diagnostic microscope such as a microscope, binoculars, and NMR imaging.

 FIG. 1 shows an outline of the configuration of the signal processing device 1. The signal processing apparatus 1 includes a photographing unit 2 that captures an image of a person or the like, a control system unit 3 that drives the photographing unit 2, and a processing unit 4 that processes an image captured by the capturing unit 2. ing. In addition, the signal processing device 1 according to this embodiment further includes a recording unit 5 that records an image processed by the processing unit 4 and change factor information that causes an angular velocity sensor and other factors that cause changes such as image degradation. It has a detection unit 6 for detecting, and a factor information storage unit 7 for storing known change factor information that causes image degradation and the like. When the signal processing device 1 is applied as a device other than an image processing device, the photographing unit 2 is a receiving unit 2 that receives various input signals such as an audio signal (hereinafter, the photographing unit 2 and the receiving unit are appropriately described). 2).

 The imaging unit 2 is a part that includes an imaging optical system having a lens and an imaging element such as a CCD or C-MOS that converts light that has passed through the lens into an electrical signal. The control system unit 3 controls each unit in the signal processing device 1 such as the imaging unit 2, processing unit 4, recording unit 5, detection unit 6, and factor information storage unit 7.

 The processing unit 4 is composed of an image processing processor, and is composed of hardware such as an ASIC (Application Specific Integrated Circuit). The processing unit 4 generates a sampling frequency for detecting vibrations such as camera shake to be detected, and supplies the sampling frequency to the detection unit 6. The processing unit 4 controls the start and end of vibration detection. When the signal processing device 1 is applied as a device other than the image processing device, the receiving sensitivity of the receiving unit 2 can be changed depending on the magnitude of the input signal or the like.

 [0032] Further, the processing unit 4 may be configured not to be configured as hardware such as an ASIC but to be processed by software. In the processing unit 4, an original image data area serving as a change data area and a restored image data area serving as a restored data area are permanently arranged. The processing unit 4 also stores a maximum value “E” of the remaining energy amount of each pixel described later.

 max

The The recording unit 5 may employ magnetic recording means such as a force hard disk drive constituted by a semiconductor memory, or optical recording means using a DVD or the like. The recording unit 5 may be provided with a change data area and a restoration data area. You may try to memorize the maximum energy amount “E”!

 max

 As shown in FIG. 2, the detection unit 6 includes two angular velocity sensors that detect the speeds around the X and Y axes that are perpendicular to the Z axis that is the optical axis of the signal processing device 1. Is provided. By the way, the camera shake when shooting with the camera is the movement that moves in the X, Y, and Z directions, and the force that also rotates around the Z axis. And rotation around the X axis. These two variations are only a slight variation, and the captured image is greatly blurred. Therefore, in this embodiment, only two angular velocity sensors around the X axis and the Y axis in FIG. 2 are arranged. For the sake of completeness, an additional angular velocity sensor around the Z axis or a sensor that detects movement in the X or Y direction may be added. In addition, the sensor used may be an angular acceleration sensor that is not an angular velocity sensor.

 [0034] The factor information storage unit 7 stores a change factor information such as known deterioration factor information, for example, a point spread function calculated based on the aberration of the optical system and Z or the detected vibration. It is. The point spread function recorded by the factor information storage unit 7 is used by the processing unit 4 when restoring the original image, which is an image that has undergone changes such as degradation that was taken immediately after the calculation, for example. . Here, when the original image restoration process is executed, the original image is taken when the imaging power is turned off, when the processing unit 4 is not operating, or when the operating rate of the processing unit 4 is low. It can be a period delayed from In this case, the original image data stored in the recording unit 5 and the change factor information such as the point spread function for the original image stored in the factor information storage unit 7 are associated with each other. Stored for a long time. As described above, the advantage of delaying the timing of executing the restoration processing of the original image from the timing of shooting the original image is that the burden of the processing unit 4 at the time of shooting involving various processes can be reduced. When the signal processing device 1 is applied as a device other than an image processing device, the temperature, humidity, etc. detected by the detection unit 6 may change the reception characteristics of the reception unit 2 or the characteristics of the entire system. Therefore, they can be recorded and used as change factor information. In addition, the response characteristic function of the system that is already working, such as the impulse response of the system, can be stored in the factor information storage unit 7.

Next, the processing unit 4 of the image processing apparatus which is the signal processing apparatus 1 configured as described above. An outline of an example of the image restoration processing method performed will be described with reference to FIG.

In FIG. 3, “E” is the light energy (original image pixel energy) of each pixel of original image data Img ′ (details will be described later) in which changes such as deterioration have occurred, and the change data Stored in the original image data area. “F” is the pixel energy transferred from the original image data area as the nth change area to the restored image data area as the restored data area (hereinafter referred to as transition pixel energy). “E” is the amount of remaining energy remaining in the original image data area, which is the change data area, due to the transition of the transition energy F for the first time and n times up to the nth time, and is the energy to be processed. “R” is restored data stored in the restored image data area, which is the restored data area, and becomes approximate data of the original image data “Img” by performing the image restoration process shown in FIG. “X” is a predetermined value of the remaining energy amount E. “G” is data of change factor information (= deterioration factor information (point spread function)) detected by the detection unit 6 and is stored in the recording unit of the processing unit 4. “Ga” is the barycentric value of the point spread function of the data of the change factor information G (= degradation factor information (point spread function)) detected by the detector 6. “Img” is data of an original image, that is, an image that should have been originally taken, and is original signal data. Note that the original signal data is defined in this specification as including the original image I mg (the signal that should have been acquired) and the data before the change and the approximate signal data thereof. . Original image data Img 'refers to the data of a captured image, that is, a deteriorated image. Here, the relationship between Img and Img 'is expressed by the following equation (1).

 Img, = Img water G (1)

 Here, “*” is an operator representing a superposition integral.

“T” shown in FIG. 3 is a pixel energy amount for removing pixel energy corresponding to the transition pixel energy F from the original image data area, and is the same amount as the transition pixel energy F. “Ε” is the maximum value of the remaining energy of each pixel in the original image data area.

 max

It is. Here, it is assumed that the relationship between the remaining energy amount E and the transition pixel energy F of each pixel in the original image data area is expressed by the following equation (2). In other words, the transition pixel energy F uses the reciprocal of the centroid value Ga of the point spread function which is the data G of the change factor information, using the remaining energy amount E (energy to be processed) of each pixel in the original image data area. You can get more than that. Here, “k” is a component ratio with respect to the energy of a pixel corresponding to the specific pixel in the original image Img, which is included in the energy of a specific pixel of the captured original image data Img ′. Since “k” is unknown, it can be set arbitrarily within the range of “0 <k ≦ 1”.

 F = kX E / Ga …… (2)

[0038] The processing routine of the processing unit 4 in Fig. 3 also begins to extract the pixel energy of one element constituting the original image data Img 'as the original image pixel energy E (step S101). Here, at the current stage (n = 0), E = E (original image data) and the restored image data

 0

 Restoration data in the data area R = 0. In the first processing (n = l), first the original image pixel energy

 0

 Lugi E = E is the inverse of the center of gravity Ga of the point spread function Ga (the largest value of the change factor information data G)

 0

 Multiply the number to obtain the transition pixel energy F (step S102). Next, the transfer pixel energy F is transferred to the restored data R in the restored image data area. That is, the transition pixel energy F is added to the restored data R in the restored image data area to

 Ten

 The restored data is R (step S103).

 [0039] Next, the original image data region force is removed from the pixel energy F of the transition pixel energy F transferred to the restored image data region. In that case, the point spread function which is the change factor information data G is used. This is because the deterioration of the data is caused by passing through a filter called change factor information data G, and the transition pixel energy 1 F can be removed from the original image data area force so that there is no contradiction before and after the deterioration. Because. Therefore, the pixel energy amount T obtained by the superposition integration (step S104) of the transition pixel energy F and the change factor information data G is removed from the original image data area, and the residual energy in the original image data area is removed. The remaining energy amount E is obtained (step S 105).

 [0040] The above steps S102, S103, S104 and S105 are sequentially performed on all remaining pixels constituting the original image data Img in the original image data area. At this time, the energy E to be processed in step S102 is the energy transfer of surrounding pixels.

 0

When the transition pixel energy F is removed from the original image data area in the row processing, the value corresponding to the pixel obtained by the multiplication integration is removed. Then, for all the pixels, the above steps S102, S103, S104 and S105 are performed. After that, the maximum value E of the remaining energy amount E in the original image data area is stored (step

 1 max

 S106).

 [0041] Next, the maximum value E of the remaining energy amount E is less than a predetermined value X, that is, the initial value.

 1 max

 Most of the original image pixel energy E = E existing in the original image data area is restored image data.

 0

 It is determined whether the restored data R in the restored image data area can be regarded as the original image data Img (step S107). In this example, the predetermined value X is set to a value close to “0” other than “0”, and it is determined whether E is less than the predetermined value X.

 In step S102, S103, S104, S1 05, S106 and S107, all of the pixels are set as “n = n + 1” = 2. In the processing procedure of FIG. 3, “n = n + 1” is performed when the processing of steps S102, S103, S104, and S105 is completed for all the pixels, and steps S106 and S107 are performed. In other words, the number of processes for the entire image is represented as n. Note that step S107 may be a step of determining how many pixel forces S of the remaining energy amount E are equal to or greater than the predetermined value X. In this case, even if there are some pixels with a residual energy amount E that is equal to or greater than the predetermined value X, the restoration data R can be regarded as being sufficiently approximate to the original image data Img if the number is small. .

[0042] max when the maximum value E of the remaining energy amount is equal to or greater than a predetermined value X in step S107.

 Processing will be described. Since E is greater than or equal to the predetermined value X, n = n + 1 and max for all pixels

 Subsequently, steps S102, S103, S104, S105, S106 and S107 are performed. In other words, the transition pixel energy F is obtained by multiplying the reciprocal of the remaining energy E 〖, the barycentric value Ga of the point spread function (the largest value of the change factor information data G) (step S102). Next, the transfer pixel energy F is transferred to the restored image data area, and new restored data R is obtained. That is, the current transition pixel energy F is added to the restored data R of the restored image data area restored up to the previous (n−1) to obtain new restored data R. Next, the pixel energy T obtained by the superposition integration (step S104) of the transition pixel energy F and the change factor information data G is removed from the original image data area, and a new remaining energy E is obtained (step S105).

[0043] The above steps S102, S103, S104 and S105 are sequentially performed on all remaining pixels in the original image data area. And the remaining energy amount in each pixel The maximum value “E” is stored (step S106). Then, it is determined whether or not the remaining energy amount maximum value E force S max max is less than a predetermined value X (step S107). If E is greater than or equal to the predetermined value X,

 max

 As “n = n + 1”, the processes of steps S102, S103, S104, S105, S106 and S107 are repeated. When E becomes less than the predetermined value X, the restored data R becomes the original image data Img.

 max n

 And the restoration process ends (step S108).

 Based on FIG. 4, the concept of the camera shake restoration processing according to this embodiment will be described below.

 Original image data Img force If the change factor information data G changes to the original image data Img ', the original image data Img' is constructed using the change factor information data G that is the same filter. If all of the original image pixel energy E in all pixels is transferred to the restored image data area, the restored data R in the restored image data area should theoretically approach the original image data Img.

 Next, details of the camera shake restoration processing method shown in FIGS. 3 and 4 (repeated processing of steps S102, S103, S104, S1 05, S106 and S107) are shown in FIGS. This will be described with reference to FIGS.

 [0046] (Restore algorithm)

 When there is no image degradation due to camera shake or the like, light energy (pixel energy) corresponding to a predetermined pixel is concentrated on that pixel during the exposure time. Also, when there is camera shake, the pixel energy is distributed to the pixels that are shaken during the exposure time. Further, if the blur during the exposure time is known, the method of dispersion of the pixel energy during the exposure time is influential, so that it is possible to create an image with no blur from the blurred image.

 [0047] Hereinafter, for the sake of simplicity, the description will be made in one horizontal dimension. The pixels are designated as S—1, S, S + 1, S + 2, S + 3,. When there is no image degradation due to blurring, etc., the pixel energy during the exposure time is concentrated on that pixel, so the pixel energy concentration is “1.0”. This state is shown in Fig. 5. In addition, as shown in Fig. 5, an example of the shooting results when there is no image degradation is shown in the table of Fig. 6. The image shown in Fig. 6 is the correct image data Img when there is no deterioration. Each data is represented by 8-bit (0 to 255) data.

[0048] Image degradation due to blurring or the like occurs during the exposure time, and 50% of the exposure time is S-th. Suppose that 30% of the time is on the S + lth pixel and 20% of the time is on the S + 2nd pixel. The distribution method of pixel energy is as shown in the table in Fig.7. This is data G of change factor information. In addition, the barycentric value Ga of the point spread function is the value of the portion where the energy is most concentrated, and is the value “0.5” of the portion that has been shifted by 50% of the exposure time.

 [0049] The blurring is uniform for all pixels, and the upper blurring (vertical blurring) is not blurred. As shown in Fig. 7, the blurring situation, that is, the pixel energy of each pixel, Specifically, as shown in FIG. 8, for example, “120” of the pixel “S-3” is “α = 0.5” of the variation factor information data G, which is the blur information. According to the dispersion ratios of “j8 = 0.3” and “γ = 0.2” (Fig. 7), “S-3” pixel is “60”, “S-2” pixel is “36”, “S — Distribute to “1” pixels as “24”. Similarly, “105” which is the pixel energy of “S-2” becomes “52.5” in “S-2”, “31.5” in “S-1”, “21” in “S”. scatter. Similarly, the pixel energy is distributed to other pixels.

 [0050] The relationship between "A", which is the energy of a specific pixel in the original image data Img, and energy "A '" of the pixel corresponding to the specific pixel in the captured original image Im g' is generally Is expressed by the following equation (3).

 A X Ga = k XA, ... (3)

Since the center-of-gravity value “Ga” of the point spread function is the value where the energy is most concentrated, in this example, as shown in FIG. 7, “α = 0.5”. In this “α”, the energy “A” of a specific pixel in the original image data Img is dispersed into the energy “Α ′” of a pixel corresponding to the specific pixel in the original image Img ′ taken due to blurring. “8 '” is dispersed from the energy of other pixels in the vicinity of “Α” by a dispersion ratio other than α (such as / 3 and γ in FIG. 7). It consists of the sum of the pixel energy. Needless to say, this equation (3) can be applied to α values other than “α = 0.5” (Fig. 7). Equation (3) can also be applied to cases where blurring occurs over the range of less than or exceeding the range of three pixels as shown in Fig. 7. Furthermore, even if the centroid Ga of the point spread function is in a portion such as “| 8” or “γ” shown in FIG. 7, Equation (3) can be applied. [0051] Next, consider how the pixel energy should be transferred from the original image data area to a specific pixel in the restored image data area. Here, since the original image data Img is restored to the restored image data area, in order to obtain the energy “A” of a specific pixel of the original image data Img, “A” of the restored image data area is obtained from the original image data area. It should move to a specific pixel corresponding to “”. Therefore, from equation (3), “A = kXA′Za” is shifted from the original image data area to a specific pixel corresponding to “A” in the restored image data area. Then, the pixel energy corresponding to AJ is removed from the original image data area. At this time, in order to remove the original image data area force pixel energy so that there is no contradiction as a whole, the data G of the change factor information is removed. For example, if the specific pixel corresponding to “A” is the pixel “S” shown in FIG. 7, the pixel energy to be removed from the pixel “S” in the original image data area is “k ΧΑ ′ / α Χ”. α, the pixel energy to be removed from the pixel “S + 1” in the original image data area is “kXA'Za X j8”, and the pixel energy to be removed from the pixel “S + 2” in the original image data area is “ kXA'Za X γ ". Then, the sum of these removed pixel energies becomes the pixel energy amount “A” transferred to the pixel “S” in the restored image data area. In this example, a specific state of pixel energy transition will be described below by setting “k = 0.8”.

 FIG. 8 and FIG. 9 show the original image pixel energy “E” shown in the first step S101 of the iterative process. In the pixel to be processed first, this original image pixel energy E is the energy E to be processed. On the other hand, in step S102, the barycentric value Ga of the point spread function

 0

Multiply the inverse of to get the transition pixel energy F. For example, when focusing on the pixel “S-3”, the center of gravity Ga of the point spread function of the original image data (photographed data) (“E” in FIG. 9) is “0.5”. Then, “0.8X60 / 0.5 = 96” is transferred from the original image data area to the pixel “S-3” in the restored image data area (step S103) (F (S-3) in FIG. 9). Then, the pixel energy of “96” is removed from the original image data area. Then, the restored image data (R (S-3) in FIG. 9) becomes “96”, “0”. Here, since the data G of the change factor information is “0.5”, “0.3”, “0.2” as described above, the pixel “S-3” in the original image data area is “96X0”. 5 = 48 ”,“ 96X0.3 = 28.8 ”from pixel“ S-2 ”, and“ 9 6X0.2.1 = 29.2 ”from pixel“ S-1 ”should be removed (FIG. 9). Then T (S—3)). Therefore, pixels “S-3” to “48”, pixels “S-2” to “28.8”, and pixels “S-1” to “19” in the original image data area. 2 ”energy is removed (step S104). At this time, the remaining energy amount E of each pixel in the original image data area is removed, the pixel “S-3” is “12”, the pixel “S-2” is “59.7”, Pixel “S-1” becomes “66.3”, and other pixels become “89”, “117”, “105”, “114”, “142” (Step S105) (“E (S — 3) ").

Next, when attention is paid to the pixel “S-2”, the energy of “59.7” remains as the energy E to be processed. If the same processing as the pixel “S-3” described above is performed, the original image

 0

 The pixel energy of “95.52” is transferred from the data area to the pixel “S-2” in the restored image data area (F (S-2) in FIG. 9), and the pixel “S—2” in the restored image data area The restored data R of “” becomes “95.52”, and the restored image data (R (S−2) in FIG. 9) becomes “96”, “95.52”, “0”,. Then, the pixels “S-2” to “47.76”, “28.656” from the pixel “S-1”, and “19.104” from the pixel “S” are removed from the original image data area. As a result, the remaining energy amount E of each pixel in the photographic data, that is, the original image data area is removed from the transition (“T (S-2)” in FIG. 9), and the pixel “S-3” Is “12”, pixel “S-2” is “11. 94”, pixel “S 1” is “37. 644”, pixel “S” is “69. 896”, and other pixels are 117, “105”, “114” and “142” (“E (S-2)” in FIG. 9).

Next, paying attention to the pixel “S-1”, the same processing as the above-described pixel “S-3” is performed. Currently, the energy of “37. 644” remains as energy E to be processed. And the original picture

 0

 The pixel energy of “60.23” is transferred from the image data area to the pixel “S-1” in the restored image data area (F (S—1) in FIG. 9), and the pixel “S— The restored data R of “1” is “60.23”, and the restored image data (R (S-1) in FIG. 9) is “96” “95.52” “60.23” “0”. · Become. Then, “30. 115” is removed from the pixels “S−1”, “18.069” from the pixel “S”, and “12. 046” from the pixel “S + 1” in the original image data area. As a result, the captured data, that is, the remaining energy amount E of each pixel in the original image data area, is removed from the transition (“T (S-1)” in FIG. 9), and the pixel “S-3” is removed. "12", pixel "S-2" force ^ 11.94 ", pixel" S-1 "" 7.529 ", pixel" S "" 51. 827 ", pixel" S + 1 " “104. 954”, and the other pixels are “105”, “114”, and “142” as before (“E (S-1)” in FIG. 9).

In this way, the pixel energy is sequentially shifted for all the pixels. And once In the processing of the eye (n = 1), the original image pixel energy E of all the pixels “S-3”, “S-2”, “S-1” ... “S + 4” does not move to the restored image data area, The remaining energy E of each pixel in the original image data area remains as a large value. The remaining energy maximum value “E” (

 n max “19.02” in the pixel “S + 4”) is stored in the memory of the control unit 4 (step S106). Whether the maximum remaining energy E is less than a predetermined value X (for example, “5” in this example).

 max

 It is determined whether or not (step S107). As a result of the above processing, E> X, so the same

 max

 The pixel energy transfer process is performed for the second time (n = 2). And then E> X

 The same transition process is repeated until max is satisfied (until the remaining energy amount E force of all pixels approaches “0”).

 Here, as a result of the specific pixel in FIG. 9 shifting the transition pixel energy F, the restoration data R may exceed a predetermined upper limit value, or the remaining energy amount E may be a negative value. . The occurrence of this condition means that the transition pixel energy F has been set to an inappropriate value. Therefore, the following explains how to review the transition value without any contradiction as a whole even in such a case.

 [0057] (Transition value review method 1)

 Pixels with a negative residual energy amount are removed at that time by using the value of the previous process as it is without transferring the transition pixel energy F that should be moved to the restored image data area. We expect that the pixel energy will not be an inappropriate value. Next time, the remaining energy amount of the surrounding pixels will decrease, and the pixel energy value to be removed from the pixels that will become negative if it moves this time, that is, the pixel energy value to be extracted will be smaller than this time. There is a high possibility of becoming “0” or more. However, for example, if the previous value is used as it is when the pixel “S” becomes negative, the remaining energy amount of the processed pixel such as the pixels “S-2” and “S-1” is set to “0”. If you get close to it, it will not be restored even if you reach it, and it will not converge. Therefore, in this case, it is desirable to review the restoration values of the pixels “S-2” and “S-1”.

 [0058] (Transition value review method 2)

The concept of the review method after migration will be explained based on the conceptual diagram shown in Fig. 10. Already recovered A part of the pixel energy transferred to the original image data area is returned to the original image data area as a correction amount (hereinafter referred to as correction energy cE), and the remaining energy amount E is set to 0 or more. At this time, the correction energy cE is superimposed and integrated with the data G of the change factor information and returned to the original image data area. Then, the transition value can be reviewed without any contradiction as a whole.

Next, the transition value review process for setting the remaining energy amount E, which is a negative value based on the above-described idea, to “0” will be described. Specifically, according to the processing routine of FIG. 3, the pixel energy transfer processing of FIG. 9 is performed in the order of the pixels “S-3”, “S-2” …… “S + 4”, and the repetition processing is continued. In the third time (when n = 3 in Fig. 3), the remaining energy amount E 1S "of the pixel" S-1 "in the original image data area during the pixel energy transfer process to the pixel" S-3 "in the restored data area -0.624 ". For reference, Fig. 11 shows the processing when n = 2 in Fig. 3, and Fig. 12 shows the processing when n = 3 in Fig. 3 (only three decimal places are displayed. The same applies to the description of FIG.13, FIG.14, and FIG.15.) ₀ Therefore, the process of setting the value of "-0.624" to "0" will be described. In this process, the pixel energy distribution shown in Fig. 7 affects the pixel "S-1" in the original image data area, and the transition value (n The following processing is performed on the assumption that (3) is inappropriate. The process will be described below based on the table in FIG.

 [0060] In order to set the remaining energy amount E (= —0. 624) of the pixel “S-1” in the original image data area to “0”, the restored data of the pixel “S-3” in the restored image data area From R (= 119.040), the correction energy cE is returned to the original image data area. Here, the ratio in which the pixel energy of the pixel “S-3” in the restored image data area is dispersed to the pixel “S-1” in the original image data area is “γ = 0. It is. Therefore, the correction energy cE is cE = —Ε / γ = — (— 0.624) /0.2.3=1.20 from cE X γ = —E.

 [0061] The correction energy cE (= 3.120) to the pixel “S-3” in the restored image data area is the pixel “S-1”, “S-2”, “S-3” in the original image data area. Thus, from the energy dispersion ratio shown in Fig. 7, it can be considered that the pixel energy was dispersed as follows.

(1) Pixel “S-3”: distributed it force 0.5 force, 3. 120 X 0.5 = 1. 560 (2) Pixel “S-2”: Distributed it force S “0.3” force, 3. 120 X 0.3 = 0. 936

 (3) Pixel “S-1”: Since the dispersion ratio is “0.2”, 3. 120 X 0.2 = 0.624

 [0062] By removing the correction energy cE (= 3.120) from the pixel "S-3" in the restored image area and returning the correction energy to the original image area according to the energy dispersion ratio shown in FIG. The transition value review process ends. In the transition value review method 2, the transition value (n = 3 times) to the pixel “S—3” in the restored image data area, which affects the pixel “S—1” in the original image data area, is inappropriate. Force of processing on the premise that there is an improper transition value (n = 3 times) to the pixel “S—2” in the restored image data area, which affects the pixel “S—1” in the original image data area Based on the assumption that it is, it can be processed based on the same idea.

 [0063] (Transition value review method 3)

 For the transition value review method 3, as with the transition value review method 2, the remaining energy amount of the pixel “S-1” in the original image data area when n = 3 in FIG. 3 E 1S is shown in the table of FIG. In this way, set “0.624” to “0”. Therefore, the pixel energy distribution shown in Fig. 7 affects the pixels "S-3" and "S-2" in the restored image data area, which affects the pixel "S-1" in the original image data area. Processes on the assumption that the migration value is inappropriate. Hereinafter, the process will be described with reference to the table of FIG.

 [0064] In order to set the remaining energy amount E (= -0.624) of the pixel "S-1" in the original image data area to "0", the restored data of the pixel "S-3" in the restored image data area From R (= 119.040), the correction energy cE derived from the following equation (4) is transferred to the original image data area:

 S-3

 return. Here, “cE” is the restored data R of the pixel “S-3” in the restored image data area.

 This is the correction energy of S- 3 n and the total energy returned to the original image data area. Also, the correction energy cE derived from the following equation (5) is returned to the original image data area from the restored data R (= 105.408) of the pixel “S-2”. Here, “cE” is displayed in the restored image data area.

 S S-2

 This is the correction energy for the restoration data R of the pixel “S-2” in this case, and the total energy returned to the original image data area.

 [0065] cE = -Ε Χ Ρ / (Ρ Χ γ + Q X β) …… (4)

 S-3 η

 cE = -Ε X Q / (P X γ + Q X j8) …… (5)

 S-2 n

Here, cE: cE = P: Q. “J8” is the pixel “S-3” in the restored image data area. ”Is the ratio of the pixel energy force S to the pixel“ _S -2 ”in the original image data area (FIG. 7), and“ γ ”is the pixel energy of the pixel“ S-3 ”in the restored image data area. This is the ratio (Fig. 7) distributed to pixel “S-1” in the original image data area.

In this example, the process proceeds with “P: Q = 1: 1”. The ratio of P and Q can be set arbitrarily. For example, P: Q = γ: | 8 can be set corresponding to the dispersion ratio of the original image data area to the pixel “S−1”. In this example, “j8 = 0.3” and “γ = 0.2”, so equations (4) and (5) are as follows.

 cE =-(-0.624) Χ1 / (1ΧΟ. 2 + 1X0. 3) = 1. 248

 S-3

 cE =-(-0.624) Χ1 / (1ΧΟ. 2 + 1X0. 3) = 1. 248

 S-2

 [0067] The correction energy cE of the pixels “S-3” and “S-2” in the restored image data area

 S-3 and cE (= 1.248) are pixels “S-3” in the original image data area before the transfer process,

 S-2

 In “S-2”, “S-1”, and “S”, the pixel energy is dispersed as follows according to the dispersion method of the pixel energy shown in FIG.

 (1) Pixel “S-3”: Since the dispersion ratio of cE is “0.5”, 1. 248X0.5 = 0.624

 S-3

 (2) Pixel “S-2”: Since the dispersion ratio of cE is “0.3” and the dispersion ratio of cE is “0.5”, 1

 S-3 S-2

 248X0. 3 + 1. 248X0. 5 = 0. 998

 (3) Pixel “S-1”: Since the dispersion ratio of cE is “0.2” and the dispersion ratio of cE is “0.3”, 1

 S-3 S-2

 248X0. 2 + 1. 248X0. 3 = 0.624

 (4) Pixel “S”: cE dispersion it force 0.2 ”, force et al. 1. 248X0.2 = 0.250

 S-2

 Then, correction energy cE and cE are respectively applied to pixels “S-3”, “S-2”, “S-1”, and “S” in the original image data area, and “return amount 1” and “return amount” in FIG. Return as the sum of 2

 S-3 S-2

 This completes the transition value review process.

 [0068] (Transition value review method 4)

The transition value review method 4 is the same as the transition value review methods 2 and 3, but the remaining energy amount E 1S of the pixel “S-1” in the original image data area when n = 3 in FIG. 3 is shown in the table of FIG. In this way, set “-0.624” to “0” or more. For this purpose, the transition value to the pixel “S-1” in the restored image data area of the previous time (n = 2 times in FIG. 3) that affects the pixel “S-1” in the original image data area “1. Review of the above transition value on the assumption that "156" is inappropriate The same processing as in methods 2 and 3 is performed. The process will be described below based on the table in FIG.

 [0069] In order to set the remaining energy E (= —0.624) of the pixel “S-1” in the original image data area to “0”, the restored data of the pixel “S-1” in the restored image data area From R, the correction energy cE is returned to the original image data area. Here, the pixel energy force of the pixel “S 1” in the restored image data area is dispersed to the pixel “S-1” in the original image data area by the formula “α = 0.5” from the energy dispersion ratio in FIG. is there. Therefore, the correction energy cE becomes cE = −E / γ = — (− 0.624) /0.5=1.248 from cE X γ = —E. Then, in the pixels “S-1”, “S”, and “S + 1” in the original image data area, the pixel energy is distributed as follows according to the distribution method of the pixel energy shown in FIG. Can be considered.

 (1) Pixel “S-1”: Since the dispersion ratio is “0.5”, 1. 248 X 0.5 = 0.624

 (2) Pixel “S”: Dispersion it force 0.3 ”, 1. 248 X 0.3 = 0.374

 (3) Pixel “S + 1”: Since the dispersion ratio is “0.2”, 1. 248 X 0.2 = 0.250

 Then, the correction energy cE (= l. 248) is removed from the pixel “S-1” in the restored image area, and the correction energy is returned to the original image area according to the energy dispersion ratio shown in FIG. The value review process ends. The advantage of this review method 4 is that the transition value can be reviewed by assuming that the transition value to the same pixel as the pixel that has less than the remaining energy E force O in the original image data area is inappropriate. is there.

[0070] (Other methods for reviewing transition values)

Review the transition values of all pixel energy transition processes that have the effect of remaining energy in the original image data area to be less than E ^ according to the review method of review methods 2, 3, and 4 above. You can also. In addition, in the above-mentioned transition value review methods 2, 3 and 4, the value of the remaining energy in the original image data area that is less than E force ^ is set to a value exceeding “0” in the process of setting it to “0”. It can also be processed. For example, when it is considered that noise is included in the original image data area, it is preferable to perform a process of setting a value exceeding “0” (a value obtained by adding noise to “0”). Furthermore, in the above-mentioned methods 2, 3, and 4 for reviewing the transition value, pixels in the original image data area when n = 3 in FIG. Remaining energy amount E _n is a negative value Restoring the previous time (when n = 3) The transition value is reviewed on the assumption that the transition value to the pixel in the image data area is inappropriate It can be done.

 [0071] During the process shown in FIG. 3, the “Do not shift! /, Process” in the above-mentioned transition value review method 1 or the “return process” in the above-described transition value review method 2, 3, or 4 Figure 16 shows a processing flowchart for explaining the processing routine related to the method for reviewing the transition value when it is executed. After step S105 in the processing flow of FIG. 3, it is determined whether the remaining energy amount E is a negative value (step S201). If E is not a negative value, the processing after step S106 in FIG. 3 is performed (step S202). If E is a negative value, the transition value review method 1, 2, 3, or 4 described above is performed (step S203), and then the process proceeds to step S202.

 [0072] In the signal processing device 1 according to the present embodiment described above, in the processing, in steps S104 and S107, the number of processing times and the criterion value for determining whether or not the remaining energy E force has been approximated to "0" in advance. Either or both can be set. For example, the number of processing can be set to any number, such as 20 or 50 times. Also, if the remaining energy E that stops processing is set to “5” in 8 bits (0 to 255) as to whether or not the remaining energy E has approximated “0”, the processing ends when it becomes 5 or less. , “0.5” can be set and the processing can be terminated when the value falls below “0.5”. This set value can be set arbitrarily. When the configuration is such that both the number of processing times and the criterion value are input, it is preferable that the processing is stopped when either one is satisfied. Note that when both settings are possible, the determination reference value may be prioritized, and if the predetermined number of processes does not fall within the determination reference value, the predetermined number of processes may be repeated.

[0073] In the description of the present embodiment, the power using the data stored in the recording unit of the processing unit 4 without using the information stored in the factor information storage unit 7 is stored here. Yes Known degradation factors such as optical aberrations and lens distortions may be used. In this case, for example, in the processing method (repetitive processing) in the previous example (repetitive processing), it is preferable to perform processing by combining the blur information and the optical aberration information as one deterioration factor. Restoration processing with optical aberration information after processing with information May be performed. In addition, the factor information storage unit 7 may not be installed, and the image may be corrected or restored only by dynamic factors at the time of shooting, such as blurring, recorded in the processing unit 4.

 [0074] If the number of repetitions of processing is set automatically or fixedly on the signal processing device 1, the set number of times may be changed by the data G of the change factor information. good. For example, when the data of a certain pixel is distributed over many pixels due to blurring, the number of repetitions may be increased, and when the dispersion is small, the number of repetitions may be decreased.

 [0075] Further, when generating the restoration data R to be an output image, the data G of the change factor information may generate data that goes out of the area of the image to be restored. is there. In such a case, data that protrudes outside the area is input to the opposite side. Also, if there is data that should come from outside the area, the data may be brought from the opposite side. For example, if the data assigned to the lower pixel is generated from the data of the pixel XN1 located at the bottom of the area, the position is outside the area. Therefore, the data is assigned to the pixel XI I located at the top right above the pixel XN1. Similarly, the pixel N2 adjacent to the pixel XN1 is assigned to the topmost pixel X12 (= next to the pixel XI I) just above.

 [0076] The restoration algorithm of this embodiment has an advantage that the data area of the signal processing device 1 can be reduced. The reason is that only the original image data area and the restored image data area are necessary for the restoration process. Further, in the restoration process, the restoration algorithm of this embodiment only repeats the movement of the pixel energy using the data G of the known change factor information, so that a rapid process is possible. Also, the data area may be processed after setting a temporary data area that is not permanent.

Although the signal processing apparatus 1 in this embodiment has been described above, various modifications can be made without departing from the gist of the present invention. For example, the processing performed by the processing unit 4 may be configured by hardware that also has a component power that is configured to share a part of processing for each of the power configured by software. In addition, the change factor information data G is inferior. It includes information that simply changes the image, not just the data of the conversion factor information, and information that improves the image as opposed to deterioration. Further, in the repeated processing, the maximum value E of the remaining energy amount E that each pixel has is not compared with the predetermined value (X), and the comparison is made.

 n max

 The remaining energy amount E can be compared with the average value or the total value. By doing so, the processing speed is improved. Furthermore, the maximum value E, the average value, or the total value of the remaining energy amount E that each pixel has, and each of these values.

 max

 Comparison with a plurality of corresponding predetermined values can also be performed.

 [0078] Also, when the number of repetition processing is set automatically or fixedly on the signal processing device 1, the set number of times may be changed by the data G of the change factor information. good. For example, when the data of a certain pixel is distributed to a large number of pixels due to blurring, the number of iterations may be increased, and when the variance is small, the number of iterations may be reduced.

 In the above-described embodiment, the restoration target is taken as image data. However, these restore processing concepts and methods and review methods can be applied to any data restoration process. For example, it can be applied to restoration of audio data and earthquake wave data. Further, in the above-described embodiment, these ideas are applied to an image that varies depending on the position of the power pixel shown in the example where the image data is blurred in each place, and a non-linear image such as gamma correction. And methods and review methods can be applied.

 [0080] Each processing method and review method described above may be programmed! Further, the program may be stored in a storage medium such as a CD (Compact Disc) DVD or a USB (Universa 1 Serial Bus) memory so that it can be read by a computer. In this case, the signal processing apparatus 1 has means for reading a program in the storage medium. Further, the program may be put into a server outside the signal processing device 1 and downloaded and used as necessary. In this case, the signal processing device 1 has communication means for downloading the program in the storage medium.

Claims

The scope of the claims

 [1] From the original signal data (hereinafter referred to as “change data”) in which changes such as deterioration have occurred, the signal before the change, the signal that should have been originally acquired, or the data of those approximate signals ( Hereinafter, the signal processing apparatus having the processing unit for restoring the original signal data)), the change data area in which the change data is stored, and the signal of the restored signal for each restoration process. A restoration data area in which data (hereinafter referred to as restoration data) is stored, and the processing unit uses the change factor information data that causes the change data to use the energy of the change data. Is transferred to the restored data area, the restored data is generated, the remaining data of the changed data area remaining by the migration is replaced with the changed data, and the same processing is repeated. When the energy value of the remaining data becomes less than zero during the above repeated processing, a part of the energy already transferred to the restored data area is used by using the change factor information. Thus, the process of returning to the change data area is performed so that the energy value of the remaining data becomes zero or more! The data is formed in the restored data area when the repetition process is completed while the repetition process is completed. A signal processing device characterized in that the original signal data is used.

 2. The signal processing device according to claim 1, wherein when the energy value of the remaining data is less than zero, processing is performed so that the value becomes zero or more.

[3] At the time of the iterative process, the energy that shifts to the restored data area is added to the restored data that has already been stored in the restored data area, and the energy value of the remaining data is calculated. 3. The signal processing device according to claim 1, wherein processing for approaching zero in a range of zero or more is performed.

[4] A signal processing apparatus having a processing unit for restoring original signal data having a plurality of element forces from change data having a plurality of element forces.

A change data area in which the change data is stored, and a restoration data area in which the data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process. , Change the element energy in one element of the above change data, Using the centroid value of the response characteristic function included in the change factor information data that causes the change factor information, the shift is made from the change data region to the restored data region, and the element energy corresponding to the transferred element energy is calculated. The process of excluding the change factor information data from the change data area is performed, the process for this one element is also sequentially performed for other elements, and the restore data is generated in the restore data area. The remaining data of the change data area remaining after the exclusion is replaced with the change data, and the same processing is repeated for each element, and the element energy transferred to the restoration data area is repeated each time it is repeated. In addition to the above, a process for generating new restored data is performed, and in the course of these series of processes, any element element of the remaining data is processed. When the energy value is less than zero, the element energy value that is less than zero is greater than or equal to zero by using the change factor information for a part of the element energy that has already moved to the restored data area. The above-mentioned series of processing is performed while performing processing to return to the change data area, and the remaining data is processed to approach zero in a range of zero or more, and is formed in the restored data area at the end of processing. A signal processing apparatus, wherein the restored data is the original signal data.

 [5] The processing unit performs a process of stopping when the energy value of the remaining data is equal to or less than a predetermined value in a range of zero or more or smaller than the predetermined value when generating the restored data. The signal processing device according to any one of claims 1 to 4.

 6. The processing unit according to any one of claims 1 to 4, wherein, when generating the restored data, the processing unit performs a process of stopping when the number of times the restored data is generated reaches a predetermined number. The signal processing apparatus as described.

 [7] In the generation of the restoration data, the processing unit is configured to generate the restoration data at a predetermined number of times, or when the number of times the restoration data is generated reaches a predetermined number, 5. The process according to any one of claims 1 to 4, characterized in that if it is smaller than /, it stops, and if it exceeds the predetermined value or exceeds the predetermined value, it repeats a predetermined number of times. Signal processing device.

[8] A signal processing apparatus having a processing unit for restoring original signal data having a plurality of element forces from change data having a plurality of element forces. A change data area in which the change data is stored, and a restoration data area in which the data of the restored signal (hereinafter referred to as restoration data) is stored for each restoration process. The energy in one element of the change data is transferred from the change data area to the restored data area using the change factor information data that causes the change, and corresponds to the transferred energy. The energy is excluded from the change data area power using the change factor information data, and the process for the one element is sequentially executed for all the other elements, and the restoration data area is restored to the restoration data area. If the value of the remaining data in the change data area remaining after the transition is less than or equal to a predetermined value in the range of zero or more or smaller than the predetermined value The processing is stopped, the restored data is treated as the original signal data, and the remaining data value is greater than the predetermined value or greater than or equal to the predetermined value, the same processing is performed by replacing the remaining data with the change data. Each time, the element energy that shifts to the restoration data area is added to the restoration data to generate new restoration data, and any of the remaining data is shifted. When the element energy value is less than zero, the element energy value that is less than zero in the remaining data is obtained by using a part of the element energy that has already been transferred to the restored data area, using the change factor information. A signal processing apparatus, wherein the remaining data amount is compared with the predetermined value while performing the process of returning to the change data area so as to achieve the above.

 9. The signal processing device according to claim 8, wherein the processing unit performs a process of stopping when the restoration data is generated when the number of times the restoration data is generated reaches a predetermined number.

 [10] When the restoration data is generated, the processing unit determines whether one or more of a maximum value, an average value, or a sum value of the remaining data values of each element has the predetermined value. 9. The signal processing apparatus according to claim 8, wherein comparison is performed.

[11] In the restoration process, when the restoration data is generated once or a plurality of times, the restoration data area is generated before the time when any one of the element energy values of the remaining data becomes less than zero. 9. The signal processing device according to claim 4 or 8, wherein the signal processing device is applied to the element energy transferred to the target. 12. The signal processing apparatus according to claim 1, wherein the signal data is image data.