JP2008116671A - Signal processing device and signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To forecast, estimate or evaluate the restored state of signal data concerning signal data which is restored to signal data considered as non-deteriorated one from signal data which is changed, such as, deteriorated. <P>SOLUTION: The signal processing device has a processing part 4 generating data on a signal before change or an original signal to be acquired originally or restored signal data that is approximate signal data to the original signal from the original signal data which is changed, such as, deteriorated by utilizing data on change factor information when changing. The processing part 4 generates already changed known data by changing prescribed known signal data by utilizing the data on the change factor information, acquires restored data for reference obtained by restoring the already changed known data to the known signal data or the signal data approximate thereto by utilizing the data on the change factor information, and contrasts the known signal data with the restored data for reference. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a signal processing apparatus and a signal processing method.

  Conventionally, when a subject is photographed by a signal (image) processing device such as a camera, it is known that the image sometimes deteriorates. Factors of image degradation include camera shake during shooting, various aberrations of the optical system, lens distortion, and the like.

  In order to correct a photographed image that has deteriorated due to camera shake during photographing, a method of moving a lens and a method of circuit processing are known. For example, as a method of moving a lens, a method is known in which camera shake is detected and corrected by moving a predetermined lens in accordance with the detected camera shake (see Patent Document 1). In addition, as a circuit processing method, a change in the optical axis of the camera is detected by an angular acceleration sensor, a transfer function indicating a blurring state at the time of shooting is acquired from the detected angular velocity, etc., and the acquired transfer function is obtained for a shot image. Is known that corrects a deteriorated image by performing inverse transformation (see Patent Document 2).

  In addition to general captured images, it is known that various images such as X-ray photographs and microscopic images are deteriorated or changed due to blurring or other causes.

JP-A-6-317824 (see abstract) Japanese Patent Laid-Open No. 11-24122 (see abstract)

  Not only in the case of correction of a deteriorated image described in Patent Documents 1 and 2, but generally, how much can be restored when correcting (hereinafter referred to as restoration) a signal that has not been deteriorated from a deteriorated signal. It is difficult to evaluate This is because the natural landscape cannot be grasped as an image signal, etc., even if it is acquired as an image signal and the image signal deteriorated due to camera shake at the time of acquisition is restored. This is because it is impossible to make a strict comparison with the actual natural scenery, and the criteria for the degree of restoration become ambiguous. In addition, the scenery in the natural world changes every moment, and the image changes depending on the exposure at the time of shooting. For this reason, the ideal image that should have been originally taken cannot be specified, and the restored image cannot be evaluated, the restored image can be compared with the ideal image, the state of the restored image can be predicted, and the like. This is true not only for image signals but also for all signals such as audio signals.

  Accordingly, an object of the present invention is to predict, estimate, or evaluate the restoration state of signal data restored from signal data that has undergone changes such as deterioration to signal data that is considered not to have deteriorated.

  In order to solve the above-described problems, the signal processing apparatus of the present invention is configured to obtain a signal before changing or a signal that should have been originally acquired (hereinafter referred to as an original signal) from original signal data in which a change such as deterioration has occurred. It has a processing unit that generates restored signal data that is data or approximate signal data of the original signal using data of change factor information when changing, and the processing unit changes predetermined known signal data Changed data is generated using the factor information data, and changed known data is generated, and the changed known data is restored to known signal data or signal data approximated by using the data of the change factor information. It has a function of acquiring reference restoration data and comparing known signal data with reference restoration data.

  According to the present invention, signal restoration for restoring original signal data to original signal data or signal data approximate thereto is performed using known signal data and reference restoration data. Here, the known signal data corresponds to the original signal data, and the reference restoration data corresponds to the original signal data. Then, the known signal data is compared with the reference restoration data. By performing this comparison, the restoration state of the restoration signal data can be predicted, estimated, or evaluated. Here, “control” refers to comparing two things together.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit evaluates the restoration state of the restoration signal data based on the result of the comparison. By adopting this configuration, the restoration state of the restoration signal data can be evaluated.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit determines, changes, or improves the generation method of the restored signal data based on the result of the comparison. By adopting this configuration, it is possible to select, change, or improve the method of generating restored signal data that is considered optimal.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit causes a person who operates the signal processing device to recognize the generation state of the restored signal data based on the result of the comparison. By adopting this configuration, it is possible for a person who operates the signal processing apparatus to grasp the generation state of the restoration signal data such as restoration accuracy.

  In the signal processing device according to another invention, in addition to the above-described invention, when the reference restoration data and the known signal data are the same or similar as a result of the comparison, the processing unit determines that the restoration signal data is the original signal data or Judged as approximate signal data of the original signal. By adopting this configuration, when it can be evaluated that the known signal data and the reference restoration data are the same or similar, it can be determined that the original signal and the restoration signal data are the same or similar. When it can be evaluated that the known signal data and the reference restoration data are far apart, it can be determined that the original signal and the restoration signal data are far apart.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit uses the data of the change factor information to generate comparison data from arbitrary signal data, and to process the original signal to be processed. The restoration data is generated by distributing the difference data between the data and the comparison data to any signal data using the data of the change factor information, and this restoration data is used instead of any signal data. Repeat processing is repeated to repeat the same processing.

  By adopting this configuration, since the original signal data is generated only by generating predetermined data, there is almost no increase in hardware and the apparatus is not increased in size. Further, a process of creating comparison data from the restored data and allocating the difference data between the comparison data and the original signal data to be processed to arbitrary signal data using the data of the change factor information is repeated gradually. Thus, the restored signal data close to the data of the original signal is obtained, so that the signal processing apparatus having a realistic circuit processing method can be obtained in restoring the signal.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit determines whether to continue or end the repeated processing based on the result of the comparison. By adopting this configuration, when it can be evaluated that the original signal data and the restored signal data are sufficiently the same or similar, it is possible to reduce the burden on the processing unit without performing unnecessary repetition processing.

  In the signal processing device according to another invention, in addition to the above-described invention, the processing unit performs a comparison using a difference between the known signal data and the reference restoration data. By adopting this configuration, the control means can be a simple numerical comparison, facilitating the processing of the processing unit.

  In addition to the above-described invention, the signal processing apparatus according to another invention has known signal data having a smaller capacity than the original signal data. By adopting this configuration, it is possible to reduce the processing load of the processing unit at the time of control and to speed up the processing.

  In addition to the above-described invention, a signal processing apparatus according to another invention uses original signal data as image data. By adopting this configuration, when image degradation occurs due to camera shake, the original image data that is the original image or the image that should have been acquired from the original image that has undergone changes such as degradation. Alternatively, the approximate image data of the original image can be restored by the restoration means.

  In order to solve the above problems, the signal processing method of the present invention acquires data of change factor information when changing from an original signal to original signal data in which a change such as deterioration has occurred, and predetermined known signal data Is obtained using changed factor information data, and the changed known data is restored to known signal data or approximate signal data using the changed factor information data. The reference restoration data is obtained, and the known signal data is compared with the reference restoration data.

  According to the present invention, signal restoration for restoring original signal data to original signal data or signal data approximate thereto is performed using known signal data and reference restoration data. Here, the known signal data corresponds to the original signal data, and the reference restoration data corresponds to the original signal data. Then, the known signal data is compared with the reference restoration data. By performing this comparison, the restoration state of the restoration signal data can be predicted, estimated or evaluated.

  In addition to the above-described invention, the signal processing method according to another invention is performed using a difference value between known signal data and reference restoration data. By adopting this configuration, the control means can be a simple numerical comparison, facilitating the processing of the processing unit.

  In the present invention, it is possible to predict, estimate, or evaluate the restoration state of signal data restored from signal data that has undergone changes such as deterioration to signal data that is considered not to have deteriorated.

  Hereinafter, a signal processing apparatus 1 according to an embodiment of the present invention that employs a signal processing method according to an embodiment of the present invention will be described with reference to the drawings. Although this signal processing device 1 is a consumer camera as an image processing device, it may be a camera for other uses such as a surveillance camera, a television camera, a handy type video camera, an endoscope camera, The present invention can also be applied to devices other than cameras, such as microscopes, binoculars, and diagnostic imaging apparatuses such as NMR imaging.

  FIG. 1 shows an outline of the configuration of the signal processing apparatus 1. The signal processing apparatus 1 includes a photographing unit 2 that captures an image of a person or the like, a control system unit 3 that drives the photographing unit 2, and a processing unit 4 that processes an image captured by the photographing unit 2. is doing. The signal processing apparatus 1 according to this embodiment further includes a recording unit 5 that records an image processed by the processing unit 4 and an angular velocity sensor, and detects change factor information that causes a change such as image degradation. And a factor information storage unit 7 for storing change factor information that causes image degradation and the like. Note that when the signal processing device 1 is applied as a device other than an image processing device, the photographing unit 2 is a receiving unit 2 that receives various input signals such as an audio signal (hereinafter, the photographing unit 2 and the receiving unit 2 are appropriately used). And will be used properly.)

  The photographing unit 2 includes a photographing optical system having a lens and a photographing element such as a CCD or C-MOS that converts light passing through the lens into an electric signal. The control system unit 3 controls each unit in the signal processing device, such as the photographing unit 2, the processing unit 4, the recording unit 5, the detection unit 6, and the factor information storage unit 7.

  The processing unit 4 is configured by an image processing processor, and is configured by hardware such as an ASIC (Application Specific Integrated Circuit). The processing unit 4 generates a sampling frequency for detecting vibrations such as camera shake to be detected and supplies the sampling frequency to the detection unit 6. The processing unit 4 controls the start and end of vibration detection. When the signal processing device 1 is applied as a device other than the image processing device, the reception sensitivity of the receiving unit 2 can be changed according to the magnitude of the input signal or the like.

  The processing unit 4 may store image data or predetermined known image data as a base when generating comparison data to be described later. Further, the processing unit 4 is not configured as hardware such as an ASIC, but may be configured to perform processing by software. The recording unit 5 is composed of a semiconductor memory, but magnetic recording means such as a hard disk drive or optical recording means using a DVD or the like may be employed.

  As shown in FIG. 2, the detection unit 6 detects speeds around the X axis and the Y axis that are perpendicular to the Z axis that is the optical axis of the signal processing apparatus 1. If there is a camera shake at the time of shooting, the shot image becomes a blurred image. Such camera shake also causes movement in each direction in the X, Y, and Z directions and rotation around the Z axis. However, the most affected by each variation is the rotation around the Y axis and the X axis. Rotation around. These two variations are only slightly varied, and the captured image is greatly blurred. For this reason, in this embodiment, a PITCH (motion in the vertical (Y) direction) detection sensor and YAW (motion in the left and right (X) direction) are used to detect blurring around the X axis and the Y axis in FIG. 3) a detection sensor and a ROLL (tilt in the left-right (X) direction) detection sensor. However, for the sake of completeness, an angular velocity sensor around the Z axis may be further added, or a sensor for detecting movement in the X direction or the Y direction may be further added. The sensor used may be an angular acceleration sensor instead of an angular velocity sensor. When the signal processing apparatus 1 is applied to a signal other than an image signal and the reception characteristic or the response characteristic of the signal processing system is affected by, for example, temperature or humidity, the detection unit 6 includes a thermometer or A hygrometer can be included. In this manner, the detection unit 6 observes data of change factor information.

  The factor information storage unit 7 is a recording unit that stores change factor information such as deterioration factor information, such as a point spread function calculated based on aberrations of the optical system and / or detected vibrations. The point spread function recorded in the factor information storage unit 7 is, for example, from the original image data (image data that has undergone changes such as deterioration) taken immediately after the calculation to the original image (the image before the change or the original image) This is used by the processing unit 4 in the process of restoring the image) that should have been performed. When the signal processing device 1 is applied as a device other than an image processing device, the temperature, humidity, etc. detected by the detection unit 6 may change the reception characteristics of the reception unit 2 and the characteristics of the entire system. They can be recorded and used as change factor information. Further, the response characteristic function of the system that is known in advance, such as the impulse response of the system, can be stored in the factor information storage unit 7.

  Here, the time when the original image data is restored to the original image or its approximate image data is when the processing power supply is turned off, when the processing unit 4 is not operating, and when the processing unit 4 is operating. When the original image is low, it can be delayed from the time when the original image was taken. In that case, the original image data stored in the recording unit 5 and the change factor information such as the point spread function for the data stored in the factor information storage unit 7 are associated with each other. Stored for a long time. As described above, the advantage of delaying the timing of executing the restoration processing of the original image data from the timing of shooting the original image is that the burden on the processing unit 4 at the time of shooting involving various processes can be reduced.

  Next, an outline of the signal processing device 1 and the signal restoration method (signal processing method) according to the present embodiment configured as described above will be described based on the drawings.

(First signal restoration method)
As a first signal restoration method, a classic restoration filter such as a Wiener filter is given. This Wiener filter is a least square filter. This Wiener filter can obtain a good restored image if the magnitude and direction of filter shake (noise) are known. In addition to the Wiener filter, a general inverse filter, a restricted least square filter, and the like can be referred to as a classic restoration filter. These represent changes due to the response characteristics of the signal processing system as a transfer function, and the output signal, which is an observation signal obtained through the signal processing system, is restored to an input signal with no change or a signal approximate to this using the transfer function. It is a technique. The classic restoration filter has an advantage that the processing time is short.

(Second signal restoration method)
The second signal restoration method is an iterative process. In the iterative processing, when the signal restoration means is executed, the processing unit 4 generates comparison data from arbitrary signal data using the data of the change factor information, and compares the original signal data to be processed with the comparison data. The restoration data is generated by allocating the difference data to the arbitrary signal data using the data of the change factor information, and using the restored data instead of the arbitrary signal data, the same processing is performed. Repeated repetitive processing is performed to bring the restored data closer to the original signal data.

The outline of the iterative process will be described with reference to FIG. FIG. 3 is a processing flowchart for explaining a processing routine related to signal (image) restoration means (iterative processing). In FIG. 3, “I ₀ ” is an arbitrary initial image and is image data stored in advance in the recording unit of the processing unit 4. “I ₀ ′” indicates change image data of the initial image data I ₀ , and is comparison data for comparison. “G” is a change function calculated from data of change factor information (= deterioration factor information (point image function)) detected by the detection unit 6 and is stored in the recording unit of the processing unit 4. “Img ′” is data of the original image.

“Δ” is data of the difference between the original image data Img ′ and the comparison data I ₀ ′ (hereinafter referred to as first difference data). “K” is an allocation ratio based on the data of the change factor information. “I _{0 + n} ” is restored image data (restored data) newly generated by allocating the first difference data δ to the initial image data I ₀ based on the data G of the change factor information. “Img” is data of the original image. Here, the relationship between Img and Img ′ is represented by the following equation (1).
Img ′ = Img * G (1)
Here, “*” is an operator representing a superposition integral. same as below.

Processing routine of the processing unit 4 first begins to prepare any image data _{I 0} (step S101). As the initial image data I ₀ , the changing original image data Img ′ may be used, and any image data such as black solid, white solid, gray solid, or checkered pattern may be used. good. In step S102, data I ₀ of an arbitrary image that is an initial image is inserted instead of Img in the equation (1), and comparison data I ₀ ′ that is a changed image is obtained. Next, the original image data Img ′ and the comparison data I ₀ ′ are compared to calculate first difference data δ (step S103).

In step S104, it is determined whether or not the absolute value of each of the first difference data δ is less than a predetermined value. If the absolute value is greater than or equal to the predetermined value, new restored image data (= restored data) is generated in step S105. Perform the process. That is, the first difference data δ when the original image data Img ′ and the comparison data I ₀ ′ are compared is distributed to arbitrary image data I ₀ based on the change function G, and new restored data I _{0 + 1} is obtained. Generate. Thereafter, steps S102, S103, S104, and S105 are repeated.

In step S104, if the absolute value of the first difference data δ is less than the predetermined value, the iterative process is terminated. Then, the restored data I _{0 + n} at the time when the repetitive processing is completed is estimated as the original image data Img. That is, when the absolute value of the first difference data δ is smaller than a predetermined value, the restored data I _{0 + n that} is the basis of the comparison data I _{0 + n} ′ is very close to the original image data Img. The restored data I _{0 + n} is estimated as the original image data Img. The recording unit 5 may record the initial image data I ₀ and the change function G and pass them to the processing unit 4 as necessary.

  The concept of the above-described iterative processing method is summarized as follows. That is, in this processing method, the processing solution is not solved as an inverse problem, but is solved as an optimization problem for obtaining a rational solution. When solving as an inverse problem, it is theoretically possible, but it is difficult as a real problem.

When solving as an optimization problem, the second signal restoration method is based on the following conditions.
That is,
(1) The output corresponding to the input is uniquely determined.
(2) If the compared outputs are the same, their inputs are the same.
(3) The solution is converged by iteratively processing while updating the input so that the compared outputs are the same.

In other words, as shown in FIGS. 4A and 4B, if comparison data I ₀ ′ (I _{0 + n} ′) that is approximate to the original image data Img ′ can be generated, the original data of the generation is generated. The initial image data I ₀ or the restored data I _{0 + n} is approximate to the original image data Img.

  Next, details of the camera shake restoration processing method (repetitive processing of steps S102, S103, S104, and S105) shown in FIG. 3 will be described in detail with reference to FIGS. 5, 6, 7, 8, 9, 10, and 11. This will be described with reference to FIG.

(Image restoration algorithm)
When there is no camera shake, the light energy corresponding to a given pixel is concentrated on that pixel during the exposure time. In addition, when there is camera shake, light energy is dispersed to pixels that are shaken during the exposure time. Further, if the blur during the exposure time is known, it is possible to know how to disperse the energy during the exposure time, so that it is possible to create a blur-free image from the blurred image.

  Hereinafter, for the sake of simplicity, the description will be made in one horizontal dimension. The pixels are designated as S-1, S, S + 1, S + 2, S + 3,. When there is no blur, the energy during the exposure time is concentrated on the pixel, so the energy concentration is “1.0”. This state is shown in FIG. The imaging results at this time are shown in the table of FIG. What is shown in FIG. 6 is the correct image data Img when no deterioration occurs. Each data is represented by 8-bit (0 to 255) data.

  It is assumed that there is blurring during the exposure time, 50% of the exposure time is blurred to the Sth pixel, 30% of time is shifted to the S + 1th pixel, and 20% of time is shifted to the S + 2th pixel. The way of energy dispersion is as shown in the table of FIG. This becomes the data G of the change factor information.

  The blur is uniform for all pixels and is recognized as a linear problem. If there is no upper blur (vertical blur), the blur situation is as shown in the table of FIG. The data shown as “blurred image” in FIG. 8 becomes the data Img ′ of the degraded original image. Specifically, for example, “120” of the pixel “S-3” is in accordance with the distribution ratio of “0.5”, “0.3”, “0.2” of the data G of the change factor information that is the blur information, Dispersed in such a manner that “60” is distributed to the “S-3” pixel, “36” is distributed to the “S-2” pixel, and “24” is distributed to the “S-1” pixel. Similarly, “60” which is the pixel data of “S-2” is distributed as “30” in “S-2”, “18” in “S-1”, and “12” in “S”. The original image data Img is calculated from the deteriorated original image data Img 'and the change factor information data G shown in FIG.

The arbitrary image data I ₀ shown in step S101, can be adopted also What, When this description, using the original image data Img '. That is, the process starts with I ₀ = Img ′. In the table of FIG. 9, “input” corresponds to the initial image data I ₀ . This data I _0, that is, Img ′ is superposed and integrated with the data G of the change factor information in step S102. That is, for example, “60” of the “S-3” pixel of the initial image data I ₀ is “30” for the S-3 pixel, “18” for the “S-2” pixel, “12” is assigned to each pixel of “−1”. The other pixels are similarly distributed, and comparison data I ₀ ′ shown as “output I ₀ ′” is generated. Therefore, the first difference data δ in step S103 is as shown in the bottom column of FIG.

Thereafter, in step S104, it is determined whether or not the absolute value of the first difference data δ of each of the plurality of pixels constituting the original image data Img ′ and the comparison data I ₀ ′ is less than a predetermined value. If the result of the determination is “N”, the process proceeds to step S105. That is, the first difference data δ is distributed to arbitrary image data I ₀ using the change factor information data G to generate restored data I _{0 + n} shown as “next input” in FIG. In this case, since this is the first time, it is represented as I _{0 + 1 in} FIG.

The distribution of the first difference data δ is, for example, “15” obtained by multiplying the pixel data “30” of “S-3” by 0.5, which is the distribution ratio of the place (= “S-3” pixel). The distribution is made to the pixel “S-3”, and the data “15” of the pixel “S-2” is multiplied by 0.3 which is the distribution ratio that should have come to the pixel “S-2”. 4.5 ”is allocated, and the data“ 9.2 ”of the pixel“ S-1 ”is multiplied by 0.2 which is the distribution ratio that should have come to the pixel“ S-1 ”. Allocate “1.84”. The total amount (update amount) allocated to the pixels of “S-3” is “21.34”, and this value is added to the data I _{0 of the} initial image (here, the original image data Img ′ is used) Restored data I _{0 + 1} is generated.

As shown in FIG. 11, the restored data I _{0 + 1} becomes the input image data (= initial image data I ₀ ) in step S102 in FIG. 3, step S102 is executed, and the process proceeds to step S103. One difference data δ is obtained. Thereafter, the determination in step S104 is made in the same manner as described above. As a result of the determination, if “N”, the process proceeds to step S105, where the new first difference data δ is distributed to the previous restored data I _{0 + 1} to generate new restored data I _{0 + 2} (see FIG. 12). Thereafter, new comparison data I _{0 + 2} ′ is generated from the restored data I _{0 + 2} by performing step S102. As described above, after steps S102 and S103 are executed, if the determination in step S104 is “N”, the process proceeds to step S105. Such a process is repeated.

  As described above, the first difference data δ gradually decreases as a result of repeated processing. When the first difference data δ becomes smaller than a predetermined value, the determination in step S104 is “Y”, and the original image data Img that is not blurred is determined. can get.

  In the camera shake restoration processing method shown in FIG. 3 described above (repetitive processing of steps S102, S103, S104, and S105), the processing performed by the processing unit 4 is configured by software. You may comprise with the hardware which consists of components which performed the process of the part in a shared manner. Further, the change factor information data G includes not only the deterioration factor information data but also information for simply changing an image and information for improving an image contrary to deterioration.

  The number of processing iterations may be set automatically or fixedly on the signal processing device 1 side. In that case, the set number of times may be changed by the data G of the change factor information. For example, when the data of a certain pixel is distributed to a large number of pixels due to blurring, the number of repetitions may be increased, and when the dispersion is small, the number of repetitions may be decreased.

  Furthermore, the process may be stopped when the first difference data δ diverges during the iterative process or when the energy of the image data after the energy has moved does not decrease but increases. For example, it is possible to adopt a method of determining whether or not the light is diverging by observing the average value of the first difference data δ and determining that the light is diverging if the average value is larger than the previous value. In addition, during an iterative process, if an input is to be changed to an abnormal value, the process may be stopped. For example, in the case of 8 bits, if the value to be changed is a value exceeding 255, the processing is stopped. Further, during an iterative process, when an input that is new data is to be changed to an abnormal value, the value may not be used but may be set to a normal value. For example, when a value exceeding 255 within the 8-bit range of 0 to 255 is used as input data, it is processed as a maximum value of 255.

  In addition, when generating restoration data to be an output image, depending on the data G of the change factor information, there may be data that goes out of the area of the image to be restored. In such a case, data that protrudes outside the area is input to the opposite side. Also, if there is data that should come from outside the area, it is preferable to bring that data from the opposite side. For example, when data allocated to a lower pixel is generated from the data of the pixel XN1 positioned at the bottom in the area, the position is outside the area. Therefore, the data is processed to be allocated to the pixel X11 located at the top right above the pixel XN1. Similarly, the pixel N2 adjacent to the pixel XN1 is assigned to the topmost pixel X12 (= next to the pixel X11) directly above.

  In the second signal restoration method, the sampling frequency of the angular velocity detection sensor is set within 60 Hz to 240 Hz. However, the angular velocity may be detected every 5 μsec so that a high frequency can be detected. In addition, in this embodiment, the value serving as the determination criterion for the first difference data δ is “6” when each data is represented by 8 bits (0 to 255). That is, when it is less than 6, that is, 5 or less, the processing is finished. Further, the shake raw data detected by the angular velocity detection sensor does not correspond to the actual shake when the sensor itself is insufficiently calibrated. Therefore, in order to deal with actual blurring, when the sensor is not calibrated, correction is required to multiply the raw data detected by the sensor by a predetermined magnification.

  The iterative process which is the second signal restoration method has an advantage that a restoration result more satisfactory than the first signal restoration method may be obtained. Also, among the signal restoration methods belonging to either the first or the second signal restoration method, the quality and processing time of the restoration result differ depending on how the parameters used are set. Therefore, depending on how the parameter setting is performed, it may be selected whether or not it is an optimal signal restoration method.

  Thus, signal restoration is characterized by each method, and there are very effective situations and weak situations. It is preferable that the signal restoration can be completed in as short a time as possible. In addition, it is preferable to select and use a memory that requires less memory for the restoration process and can obtain a higher quality restoration result.

  Therefore, simulation of signal restoration, which is a kind of signal restoration method, is performed using known signal data. That is, it is determined in advance whether the restoration result of the original signal data can be sufficiently satisfied by comparing the known signal data with the reference restoration data. If it is determined that the signal is satisfactory, the original signal is restored by the signal restoration method. However, if it is determined that a satisfactory result cannot be obtained, use a different signal restoration method, or change the restoration process parameters, evaluate the restoration result of the known signal again, and verify that the signal restoration method is effective. Make a judgment. By doing so, the restoration process can be performed by the most effective signal restoration method. In addition, the quality of the restoration result can be predicted or evaluated. For example, the processing unit 4 may select one or more of the following five types of signal restoration methods, that is, the first, second, third, fourth, and / or fifth simulation methods (signal restoration methods). I do.

(First simulation method and subsequent restoration process)
The first simulation method shown in FIG. 13 and the subsequent restoration process will be described. First, the known signal data in_ref is changed by the change function g calculated from the data of the change factor information obtained when the original signal data is acquired (at the time of photographing), and the known signal data in_ref is changed by the change function g. Changed known data out_ref is obtained (step S201). Three restoration algorithms, Algo_a, Algo_b, and Algo_c, are prepared. The processing cost of these algorithms (for example, the time required for the processing) is Algo_a <Algo_b <Algo_c. The first simulation method is a part of what is processed according to the processing flow of FIG. 13, and out_ref is restored by Algo_a, Algo_b, or Algo_c. The first simulation method is completed by comparing the obtained reference restoration data dec_a, dec_b, or dec_c with In_ref. In the processing flow of FIG. 13, if this first simulation method is developed and evaluated as acceptable, the original signal is restored with the signal restoration method. If it is judged as unacceptable, it is restored with another algorithm. Make an evaluation.

  In the flow of FIG. 13, in order to obtain a quality restoration result that can be determined to be an acceptable level without incurring a processing cost, the changed known data out_ref is first restored by Algo_a to obtain reference restored data dec_a ( Step S202). The known signal data In_ref is compared with the reference restoration data dec_a (step S203), and if the evaluation result based on a predetermined standard for the reference restoration data dec-a is an allowable level, the original signal is restored with Algo_a. This is performed (step S204). In this case, the processing cost is the smallest. When it cannot be determined that the quality of the restoration signal is an acceptable level, the changed known data out_ref is restored with Algo_b to obtain reference restoration data dec_b (step S205). The known signal data In_ref and the reference restoration data dec_b are contrasted (step S206), and if it can be determined that the evaluation result based on a predetermined standard for the reference restoration data dec-b is an acceptable level, A certain original signal data is restored by Algo_b (step S207). When it cannot be determined that the quality of the restoration signal is an acceptable level, the changed known data out_ref is restored by Algo_c, and the restoration result is set as reference restoration data dec_c (step S208). If the known signal data In_ref and the reference restoration data dec_c are contrasted (step S209), and it can be determined that the evaluation result based on a predetermined criterion for the reference restoration data dec-c is an acceptable level, A certain original signal data is restored by Algo_c (step S210). If it is not possible to determine that the quality of the restored signal is at an acceptable level, the original signal data is not restored (step S211).

  Instead of ending without performing the restoration process (step S211), the reference restoration data dec_a, dec_b, dec_c is selected with the best restoration evaluation result based on a predetermined criterion, and the algorithm is selected. It may be used to restore the original signal data. In this first simulation method, three restoration algorithms are prepared, but it may be one, two, or four or more. In addition, the processing cost is compared in ascending order to determine whether it is acceptable, but the processing of all prepared restoration algorithms is evaluated, and among them, restoration based on a predetermined standard is performed. It is possible to implement a method of adopting the one having the best evaluation result and restoring by that method.

(Second simulation method and subsequent restoration process)
Next, the second simulation method shown in FIG. 14 and the subsequent restoration process will be described. The changed known data out_ref obtained by changing the known signal data in_ref with the change function g calculated from the data of the change factor information obtained when the original signal data is acquired (at the time of photographing) is obtained in the same manner as in step S201 described above ( Step S301). When the parameter that affects the speed of the restoration process of the prepared restoration algorithm Algo_a is k, the changed known data out_ref is restored with the initial value of k prepared in advance, and the reference restoration data dec_a is obtained. Obtain (step S302). Then, the known signal data in_ref is compared with the reference restoration data dec_a (step S303). As a result of the comparison, if it can be determined that the quality of the restored signal is at an acceptable level, the restoration process of the original signal data is performed using the parameter k of the restoration process (step S304). If the result of the comparison cannot be determined to be an acceptable level as the quality of the restoration signal, the restoration process parameter k is changed (step S305), and the restoration process is performed with the new parameter k after the change (step S302). ). By repeating this, the optimum restoration process parameter k is obtained. This completes the second simulation method. Thereafter, using this parameter k, restoration processing is performed on the original signal data.

  As an example of the parameter k used in the restoration process, for example, when the restoration algorithm Algo_a is the restoration processing method by the iterative process shown in FIG. 3, an update coefficient for obtaining update data by the iteration (the coefficient in step S105 in FIG. 3) k). Here, when the update coefficient k is increased, the update value in one iteration is increased, so that the convergence speed of the value of the first difference data δ can be increased. However, if the update coefficient k is too large, the value of the first difference data δ may diverge. If the update coefficient is reduced, a good result can be obtained, but the convergence speed is slowed down, so that a larger number of iterations is required, and a longer time is required for the restoration process. Therefore, it is preferable to select the update coefficient k that can obtain a good result in a short time without divergence by the second simulation method.

(Third simulation method and restoration process)
Next, the third simulation method and restoration process shown in FIG. 15 will be described. This is applied, for example, to the process of determining the number of iterations (the number of iterations) when the restoration process by the iteration process shown in FIG. In the iterative processing described with reference to FIG. 3, the data Img ′ of the observation signal (captured image or original signal) is compared with the comparison data I ₀ ′ obtained by changing the data of the restoration signal (restored image). The iterative process is repeated until the difference data δ becomes sufficiently small. However, by performing restoration simulation with a known signal, it becomes possible to make a determination based on known signal data, so that the quality of the restoration result can be made better than before.

Figure 15 is first prepared the known signal data _{P 0} (step S401). The changed known data P ₀ ′ is calculated by changing it with the change function G calculated from the data of the change factor information obtained when acquiring the original signal data (at the time of photographing) (step S402). Then, arbitrary signal data I ₀ is prepared in order to restore the original signal data Img ′ which is a captured image. I ₀ is arbitrary signal data, but is zero here (step S403). Next, reference restoration data Q ₀ is prepared to restore the known signal data P ₀ ′. Here, also the zero reference restore data _{Q 0} (step S403). Here, when the arbitrary signal data I ₀ corresponds to the original signal data Img ′, the reference restoration data Q _{0 corresponds} to the changed known data P ₀ ′. This is preferable in terms of accuracy of the simulation method.

Next, the known signal data P ₀ is compared with the reference restoration data Q ₀ (step S404). At the present stage, “Q _{0 + n} = Q ₀ ”. If the difference value is equal to or less than a predetermined value and it can be determined that the evaluation result is an acceptable level, it is determined that the restoration quality is sufficiently high, the restoration process is terminated, and the restoration data I ₀ is converted into the original signal data Img. 'Is a restoration signal (step S405). Thereafter, every time the known signal data P ₀ is compared with the reference restoration data Q ₀ (step S 404), the third simulation method included in the processing flow of FIG. 15 ends. However, at this stage, “Q ₀ = 0”, and in most cases, the “acceptable” determination is not made in step S404, and the process proceeds to step S406 and subsequent steps without proceeding to step S405.

If it is determined as "not acceptable" in step S404, to change the arbitrary signal data I ₀ in change function G, it calculates a change has been any signal data I2 0 _'(step S406). Further, the reference restoration data Q ₀ is changed by the change function G to calculate changed reference restoration data Q ₀ ′ (step S406). Then, difference data between the original signal data Img ′ and the changed arbitrary signal data I2 ₀ ′ (hereinafter referred to as second difference data δ_Img) is calculated (step S407). Further, the difference data between the changed known data P ₀ ′ and the changed reference restoration data Q ₀ ′ (hereinafter referred to as third difference data δ_ref) is calculated (step S407). The update amount is calculated using the change function G calculated from the third difference data δ_ref and the change factor data (step S408). The update amount is added to the reference restoration data Q ₀ to be set as new arbitrary signal data Q ₁ (= Q _{0 + n} ) (step S408). Then, the update amount is calculated using the second difference data δ_Img and the change function G, and the update amount is added to the reference restoration data Q ₀ to add new arbitrary signal data I ₁ (= I _{0 + n} ). Obtain (step S408). After passing through the step S408, a new arbitrary signal data Q _1, any signal data I ₁ obtained becomes respectively "Q _0" and instead of "I _0", again, following the third simulation The method is performed.

Thereafter, the known signal data P ₀ is again compared with the new arbitrary signal data Q ₁ (step S404), and if it can be determined that the evaluation result is an acceptable level, it is determined that the restoration quality is sufficiently high. Then, the restoration process is terminated (step S405). Then, the new arbitrary signal data I ₁ (= I _{0 + n} ) is used as a restoration signal of the original signal data Img ′ (step S405). If it is determined in step S404 that the restoration evaluation result does not indicate an acceptable level, steps S406, S407, S408, and S404 are repeated until the restoration level is reached. Alternatively, the number of repetitions may be determined in advance, and if not completed so far, it may be determined that a high-quality restoration signal cannot be obtained and the processing may be terminated.

In the third simulation method shown in FIG. 15, the original signal data Img ′ is restored to the original signal Img which is an ideal image or data approximate thereto, and the known signal data P using the change information G is used. ₀ and the restoration processing for reference data Q ₀ (= Q _{0 + n} ) are performed in parallel. From the result of the restoration processing using known signal data P ₀ and reference restoration data Q ₀ (= Q _{0 + n} ), the original signal The result of the restoration process for restoring the data Img ′ to the original signal Img that is an ideal image or data that approximates the original signal Img is estimated.

(Fourth simulation method and subsequent restoration process)
In the fourth simulation method shown in FIG. 16, only restoration processing using known data is performed first, and the number of iterations until a high-quality result is obtained from the restoration results using known data is obtained. The original signal is restored later. In this way, it is possible to know how much processing time is required for the original signal restoration process before performing the original signal restoration process. If it is known in advance that a very long processing time is required, it can be determined that the restoration process is not performed or another type of restoration processing method is adopted.

The fourth simulation method, first, a known signal data _{P 0} (step S501). The changed known data P ₀ ′ is calculated by changing it with the change function G calculated from the data of the change factor information obtained when the original signal data is acquired (at the time of photographing) (step S502). Then, in order to restore the changed known data P ₀ ′, reference restoration data Q ₀ is prepared. Here, Q _{0 is set} to zero (step S503).

Here, if the known signal data P ₀ is compared with the reference restoration data Q ₀ (= Q _{0 + n} ) (step S504), and it can be determined that the quality of the restoration signal is an acceptable level, FIG. It is determined that the number n of repetitions of the repetition process shown is sufficient (step S505), and the original signal data Img ′ obtained by photographing is restored by the repetition process (one repetition number) shown in FIG. Then, the restored data I ₁ is used as a restored signal of the original signal data Img ′ (step S506). Each time this comparison (step S504) is performed, the fourth simulation method ends.

The known signal data P ₀ and the reference restoration data Q ₀ (= Q _{0 + n} ) are contrasted (step S504). If the quality of the restoration signal cannot be determined as an acceptable level, and the number of iterations n is less than the predetermined number, (step S507), the reference restoring data _{Q 0} is changed by the change function G, it calculates the restored data _{Q 0} 'for change already reference (step S509). Then, difference data (hereinafter referred to as fourth difference data δ′_ref) between the changed known data P ₀ ′ and the changed reference restoration data Q ₀ ′ (= Q _{0 + n} ′) is calculated (step S510). Then, the fourth difference data δ′_ref is distributed to the reference restoration data Q ₀ using the function of the change factor data G. At that time, the update amount is calculated using the update coefficient k, and the update amount is added to the reference restoration data Q ₀ to be set as new arbitrary signal data Q ₁ (= value of n = _{0 of} Q _{0 + n} ). (Step S511). After passing through the step S511, the arbitrary signal data Q ₁ newly obtained are treated in the place of "Q _0" again, the fourth simulation method is implemented.

Thereafter, any signal data Q ₁ newly obtained, in contrast to the known signal data P ₀ (step S504), the result is the repeat count n of the iterative process shown in FIG. 3 is equal to or less than a predetermined value Judge that 2 times is enough. Then, 'the two restoration processing on (step S505), the data Img of recovered data _{I 2} original signal' obtained original signal data Img by photographing a restoration signal (step S506). If the comparison result (step S504) cannot be determined to be an acceptable level as the quality of the restoration signal, and if the number of iterations n is less than a predetermined number (step S507), the restoration result can be judged to be an acceptable level. Until this time, the reference restoration data Q ₀ (= Q _{0 + 2} ) is changed by the change function G (step S509), and the subsequent steps S510, S511, S504, S507, and S509 are repeated. Alternatively, unless the predetermined number of repetitions is reached (step S507), a restoration signal with high quality is not obtained and restoration processing is not performed (step S508).

(Fifth simulation method)
The fifth simulation method shown in FIG. 17 will be described. FIG. 17 is a process flowchart for explaining a method of evaluating the restored data I _{0 + n} obtained by the iterative process shown in FIG. In this fifth simulation, according to the evaluation result of the restoration state of the signal, the operator who operates the signal processing device 1 can appeal to the five senses such as uttering and displaying the evaluation result of the restoration state by voice. It can be recognized by means.

“P ₀ ′” is changed known data in which the known image data P ₀ is deteriorated (changed) by the data G of the change factor information obtained when the original signal data is acquired (at the time of photographing). “Q ₀ ” is arbitrary image data. “I ₀ ” in FIG. 3 may be “Q ₀ ”. “Q ₀ ′” is reference restoration data in which arbitrary image data Q ₀ is deteriorated (changed) by the change function G. “Δ ′” is data of a difference between the changed known data P ₀ ′ and the changed reference restoration data Q ₀ ′ (hereinafter referred to as fifth difference data). “Q _{0 + n} ” is a new arbitrary signal used for the next iterative process newly generated by allocating the fifth difference data δ ′ to the reference restoration data Q ₀ based on the data G of the change factor information. It is data. “K” is an update coefficient.

The processing routine of the processing unit 4 in the fifth simulation method starts by preparing known image data P ₀ and reference restoration data Q ₀ (steps S601 and S603). And with respect to known image data P ₀ and the reference restoring data Q _0, then the convolution of the change function G respectively, to obtain a change already known image data P _{0 'and} change already reference restored data Q _0' (step S602 , S604). Then, difference data (hereinafter referred to as fifth difference data δ ′) between the changed known image data P ₀ ′ and the changed reference restoration data Q ₀ ′ is obtained (step S605).

In step S606, it is determined whether or not the number of repetition processes in FIG. 17 is the same as the number of repetition processes in image restoration in FIG. If they are not the same, the process proceeds to “N”, and in step S607, new reference restoration data Q _{0 + 1} (= Q _{0 + n} = Q ₁ ) is generated. That is, the fifth difference data δ ′ is distributed to the reference restoration data Q ₀ based on the change function G, and new arbitrary signal data Q ₁ is generated. Thereafter, a new arbitrary signal data _{Q 1} using in place of _{Q 0,} step S602, S605, S606, and repeats S607.

In step S606, when the number of repetitions in FIG. 17 is the same as the number of repetition processes in signal restoration in FIG. 3, the process of repeating steps S602, S605, S606, and S607 is terminated. At this stage, the same process as the process of estimating the restored data I _{0 + n} in FIG. 3 as the data of the original image Img was performed using the known image data P ₀ , the reference restored data Q ₀ and the change function G. It is a stage. Therefore, by comparing the known image data P ₀ with the new reference restored data Q _{0 + n} at the current stage (step S608), the extent of the restored data I _{0 + n in} FIG. 3 to the unknown original image Img. You can evaluate whether you are approaching. The fifth simulation method is completed by comparing the known image data P ₀ with the reference restoration data Q _{0 + n} . If the apparent difference between the known image data P ₀ and the new arbitrary image data Q _{0 + n} at the current stage is small by the fifth simulation method, for example, the restored data I _{0 + n in} FIG. It can be evaluated that the data is close to the data, that is, the restoration result is good. However, if the apparent difference is large, the restored data I _{0 + n} in FIG. 3 is far from the data of the original image Img, and it can be evaluated that the restored result is not good.

  The signal processing device 1 according to the present embodiment and the two basic signal restoration methods and five signal restoration methods (= simulation methods) that can be employed in the signal processing device 1 have been described above. Various modifications can be made without departing from the above. For example, the image may be restored by inverse transformation or deconvolution operation, regardless of the iterative process (second signal restoration method) shown in FIGS. 3 to 12 of the first signal restoration method. Specifically, image restoration can be realized by a technique that removes a change factor by converting change image data using a transfer function or the like representing blur. By adopting this configuration, an existing mature technique can be adopted, stable signal restoration can be performed, and the configuration of the signal processing apparatus 1 can be simplified.

The data capacities of P ₀ and Q ₀ that are known image data and the data Img ′ of the original image to be restored may be the same. However, in the third, fourth, and fifth simulation methods, in order to increase the processing speed, the data capacity of various known image data is set to 5% or 10% of the data capacity of the original image data Img ′. , 30%, 50%, 70%, or 90% is preferable. In particular, in the case of the fourth simulation method, the number of iterations n is obtained in advance by simulation (step S506 in FIG. 16), and then the original image data Img ′ is restored, so that simulation processing must be accelerated. Therefore, in the fourth simulation method, it is particularly preferable that the data volume of various known image data is set to a value smaller than the data volume of the data of various original images.

Various known image data, such as P ₀ is preliminarily several prepared, it is preferable to be able to choose the appropriate one in accordance with the subject. By using image data close to the subject, appropriate evaluation can be performed. The image data to be prepared is image data such as portrait, landscape, flower, night view, evaluation chart image (resolution chart, zone plate, etc.), for example.

Note that various kinds of difference data such as the second difference data δ_ref are restored by using an apparent difference between the changed known data P ₀ ′ and the changed reference restored data Q ₀ ′ as in step S405 of FIG. The restoration state of data I _{0 + n} is determined. However, various kinds of difference data, for example, compare the changed known data P ₀ ′ with the changed reference restored data Q ₀ ′, obtain the difference value and evaluate the restored data I _{0 + n} , May be applied to the restored data I _{0 + n} . In addition, various difference data such as the first difference data δ may be a simple difference between corresponding pixels, but generally differs depending on the data G of the change factor information. For example, the first difference data δ is expressed by the following equation (2).
δ = f (Img ′, Img, G) (2)

In the third and fourth simulation methods, the reference restoration data Q ₀ (= Q _{0 + n} ) is contrasted with the known signal data P ₀ (steps S404 and S504). In this case, the changed known data P ₀ ′ and changed reference restored data Q ₀ ′ may be used instead. However, in the simulation, it is preferable to use the known signal data P ₀ corresponding to the ideal image Img as a control because the accuracy of the simulation is improved as compared to using the changed known data P ₀ ′ as a control.

  In the present embodiment, the restoration target is image data. However, these restore processing concepts and techniques can be applied to any data restore process. For example, it can be applied to restoration of digital audio data.

  Each signal restoration method described above may be programmed. Alternatively, the program may be stored in a storage medium, such as a CD, DVD, or USB memory, and read by a computer. In this case, the signal processing device 1 may be a program stored in the storage medium placed in an external server of the signal processing device 1, downloaded as needed, and used. In this case, the signal processing apparatus 1 has communication means for downloading the program in the storage medium.

It is a block diagram which shows the main structures of the signal processing apparatus which concerns on embodiment of this invention. It is an external appearance perspective view which shows the outline | summary of the signal processing apparatus shown in FIG. 1, and is a figure for demonstrating the arrangement position of an angular velocity sensor. FIG. 10 is a process flow diagram for explaining a processing routine related to a second signal restoration method (iterative process), which is a signal (image) restoration method performed by the processing unit of the signal processing device shown in FIG. 1. It is a figure for demonstrating the concept of the processing method shown in FIG. FIG. 4 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 by taking an example of camera shake, and is a table showing energy concentration when there is no camera shake. FIG. 4 is a diagram for specifically explaining a signal (image) restoration processing method shown in FIG. 3 by taking an example of camera shake, and showing image data when there is no camera shake. FIG. 4 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 by taking hand shake as an example, and showing the dispersion of energy when hand shake occurs. FIG. 4 is a diagram for specifically explaining a signal (image) restoration processing method shown in FIG. 3 by taking an example of camera shake, and is a diagram for explaining a situation in which comparison data is generated from an arbitrary image. FIG. 3 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 by taking an example of camera shake, comparing the comparison data with the blurred original image to be processed, and comparing the first difference It is a figure for demonstrating the condition which produces | generates data. FIG. 3 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 with an example of camera shake, and illustrates a situation in which restoration data is generated by allocating difference data and adding it to an arbitrary image. FIG. FIG. 3 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 by taking hand shake as an example. New comparison data is generated from the generated restoration data, and the data and processing target are generated. It is a figure for demonstrating the condition which compares with the blurred original image and produces | generates 1st difference data. FIG. 4 is a diagram for specifically explaining the signal (image) restoration processing method shown in FIG. 3 by taking an example of camera shake, and shows a situation in which newly generated first difference data is allocated and new restoration data is generated. It is a figure for demonstrating. It is a signal restoration method executed by the processing unit of the signal processing device shown in FIG. 1, and is a process flow diagram for explaining a first simulation method and a subsequent restoration process. It is a signal restoration method executed by the processing unit of the signal processing device shown in FIG. 1, and is a process flow diagram for explaining a second simulation method and a subsequent restoration process. FIG. 9 is a signal restoration method executed by a processing unit of the signal processing device shown in FIG. 1 and is a process flow diagram for explaining a third simulation method and subsequent restoration processing. FIG. 9 is a signal restoration method executed by the processing unit of the signal processing device shown in FIG. 1, and is a process flow diagram for explaining a fourth simulation method and subsequent restoration processing. FIG. 9 is a signal restoration method executed by a processing unit of the signal processing device shown in FIG. 1, and is a processing flow diagram for explaining a fifth simulation method and subsequent restoration processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Signal processing apparatus 2 Imaging | photography part (reception part)
Reference Signs List 3 Control system unit 4 Processing unit 5 Recording unit 6 Detection unit 7 Factor information storage unit G, g Change function calculated from data of change factor information Img ′ Original signal data (captured image)
I _{0 + n} , Q _{0 + n} Restoration data Img Original signal data (signal before change or signal that should have been originally acquired)
P ₀ known signal data Q ₀ reference restoration data I ₀ Arbitrary signal data P ₀ 'changed known data Q ₀ ' changed reference restoration data

Claims (12)

From the original signal data in which a change such as deterioration has occurred, the signal before the change or the signal that should have been originally acquired (hereinafter referred to as the original signal) or the restored signal data that is the approximate signal data of the original signal, In the signal processing device having a processing unit to generate using the data of the change factor information at the time of the change,
The processing unit changes predetermined known signal data using the data of the change factor information to generate changed known data, and uses the data of the change factor information to change the known change data. A signal having a function of acquiring reference restoration data obtained by restoring data to the known signal data or signal data approximate thereto and comparing the known signal data with the reference restoration data Processing equipment.   The signal processing apparatus according to claim 1, wherein the processing unit evaluates a restoration state of the restoration signal data based on the result of the comparison.   The signal processing apparatus according to claim 1, wherein the processing unit determines, changes, or improves a generation method of the restoration signal data based on the result of the comparison.   The signal processing apparatus according to claim 1, wherein the processing unit causes a person who operates the signal processing apparatus to recognize a generation state of the restored signal data based on the result of the comparison.   When the reference restoration data and the known signal data are the same or similar as a result of the comparison, the processing unit is configured such that the restoration signal data is the original signal data or the approximate signal data of the original signal. The signal processing apparatus according to claim 1, wherein the determination is made.   The processing unit generates comparison data from arbitrary signal data using the data of the change factor information, and calculates difference data between the original signal data to be processed and the comparison data. The restoration data is generated by allocating to the arbitrary signal data using the data of the change factor information, and this restoration data is used in place of the arbitrary signal data, and the same processing is repeated repeatedly. The signal processing apparatus according to claim 1.   The signal processing apparatus according to claim 6, wherein the processing unit determines whether to continue or end the iterative processing based on the result of the comparison.   The said processing part performs the said contrast using the difference of the said known signal data and the said reconstruction data for a reference, Any one of Claim 1, 2, 3, 4, 5 or 7 characterized by the above-mentioned. A signal processing device according to 1.   The signal processing apparatus according to claim 1, wherein the known signal data has a smaller capacity than the original signal data.   The signal processing apparatus according to claim 1, wherein the original signal data is image data. Obtain the data of the change factor information when changing from the original signal to the original signal data that has undergone changes such as degradation,
Obtain changed known data obtained by changing predetermined known signal data using the data of the change factor information,
Using the data of the change factor information, obtain restored reference data for restoring the changed known data to the known signal data or signal data approximate thereto,
A signal processing method comprising comparing the known signal data with the reference restoration data.   12. The signal processing method according to claim 11, wherein the comparison is performed using a difference value between the known signal data and the reference restoration data.

Images