JP5007245B2 - Signal processing device - Google Patents

Description

  The present invention relates to a signal processing apparatus.

  Conventionally, it has been known that a signal sometimes deteriorates when it is photographed by a signal processing device such as a camera. Factors that cause signal degradation include camera shake during shooting, various aberrations of the optical system, lens distortion, and the like.

  In the past, the present applicant has proposed a method of restoring an image (signal) that has deteriorated due to camera shake at the time of photographing (see Patent Document 1). The technique described in Patent Document 1 is a technique for dealing with a position having the longest exposure time when a processing unit performs processing for generating restored data in a signal processing apparatus having a processing unit for processing a signal. By adopting this technology, when restoring a shot image that has been exposed so that the exposure time is increased in the middle or at the end of the blurring path of the shot image, the positional deviation between the shot image and the restored image can be reduced. It is possible to provide an image processing apparatus that is reduced in size and generates a restored image with a natural appearance.

  In addition to general captured images, it is known that various images and signals such as X-ray photographs and microscopic images are deteriorated or changed due to blurring or other causes.

International Publication No. 2007/077733 Pamphlet

  When image restoration processing including signal processing as described in Patent Document 1 is performed, ringing is likely to occur in the vicinity of an edge portion existing in the image. It is difficult to suppress the occurrence of ringing. In addition, a phenomenon similar to ringing in an image also occurs in a general signal.

  Therefore, an object of the present invention is to provide a signal processing apparatus that can suppress the occurrence of ringing or the like when restoring a signal.

In order to solve the above-described problem, the signal processing apparatus of the present invention is a signal before changing, a signal that should have been originally acquired, or an approximated signal thereof (hereinafter referred to as “the original signal data” in which changes such as deterioration have occurred). A processing unit for restoring the original signal), and the processing unit has restoration means for generating restoration data to be an original signal using data of change factor information that is a factor of signal change. means is a means for generating an original signal part of the data of the signal components constituting the original signal data and restore the data or by moving the whole from the original signal data, the processing section of the home position of the change factor information Then, the restoration processing is performed by setting the position at which the total sum of the kinetic energy required for the movement of the data of each signal element is the minimum value.

  According to the present invention, the total amount of movement energy of the signal element data can be reduced, and the movement amount and movement distance of the signal element data can be reduced in the process of executing the restoration means. As a result, the signal change can be reduced in the process of executing the restoration means. Then, occurrence of ringing or the like can be suppressed.

In order to solve the above-described problem, the signal processing apparatus of the present invention is a signal before changing, a signal that should have been originally acquired, or an approximated signal thereof (hereinafter referred to as “the original signal data” in which changes such as deterioration have occurred). A processing unit for restoring the original signal), and the processing unit has restoration means for generating restoration data to be an original signal using data of change factor information that is a factor of signal change. means is a means for generating an original signal part of the data of the signal components constituting the original signal data and restore the data or by moving the whole from the original signal data, the processing section of the home position of the change factor information When the minimum value of the total kinetic energy required for moving the data of each signal element is Min, the restoration energy is set to a position where the kinetic energy exceeds Min and becomes Min × 1.2 or less.

  According to the present invention, the total amount of movement energy of the signal element data is set to a small range, and the movement amount and movement distance of the signal element data can be reduced in the process of executing the restoring means. As a result, the signal change can be reduced in the process of executing the restoration means. Then, occurrence of ringing or the like can be suppressed.

  In addition to the above-described invention, the signal processing apparatus according to another invention generates the comparison data from arbitrary signal data by using the data of the change factor information that causes the processing unit to cause the signal change. The original signal data to be processed is compared with the comparison data, and the obtained difference data is distributed to arbitrary signal data using the data of the change factor information in which the origin position is set. This is means for generating reconstructed data, using the reconstructed data in place of arbitrary signal data, and repeatedly performing similar processing. If this repetitive processing is performed very many times, a signal such as an image in which no ringing has occurred can be restored. Therefore, by adopting this configuration, occurrence of ringing or the like can be more reliably suppressed.

  In the present invention, it is possible to provide a signal processing device capable of suppressing the occurrence of ringing and the like.

  Hereinafter, a signal processing device according to an embodiment of the present invention will be described with reference to the drawings. Although this signal processing device is a consumer camera, it may be a camera for other uses such as a surveillance camera, a television camera, a handy type video camera, an endoscopic camera, a microscope, a binocular, The present invention can also be applied to devices other than cameras, such as diagnostic imaging devices such as NMR imaging, and devices that form part of the image adjustment function of a printer that prints images.

  FIG. 1 shows an outline of the configuration of the signal processing apparatus 1. The signal processing device 1 includes an imaging unit 2 that captures an image of a person, a control system unit 3 that drives the imaging unit 2, and a processing unit 4 that processes an image captured by the imaging unit 2. ing. The signal processing apparatus 1 according to this embodiment further includes a recording unit 5 that records an image processed by the processing unit 4 and an angular velocity sensor, and detects change factor information that causes a change such as image degradation. And a factor information storage unit 7 for storing known change factor information that causes image degradation and the like.

  The photographing unit 2 includes a photographing optical system having a lens and a photographing element such as a CCD or C-MOS that converts light passing through the lens into an electric signal. The control system unit 3 controls each unit in the signal processing device, such as the photographing unit 2, the processing unit 4, the recording unit 5, the detection unit 6, and the factor information storage unit 7.

  The processing unit 4 is configured by an image processing processor, and is configured by hardware such as an ASIC (Application Specific Integrated Circuit). The processing unit 4 generates a sampling frequency for detecting vibrations such as camera shake to be detected and supplies the sampling frequency to the detection unit 6. The processing unit 4 controls the start and end of vibration detection. Further, the processing unit 4 detects and stores a blurring locus (history) until the start and end of the vibration detection and a time (weight) remaining at each point on the locus. And the process part 4 sets the origin position of a blurring locus so that the movement energy of the light energy mentioned later may become the minimum value.

  Further, the processing unit 4 may store image data that is a source for generating comparison data, which will be described later. Further, the processing unit 4 is not configured as hardware such as an ASIC, but may be configured to perform processing by software. The recording unit 5 is composed of a semiconductor memory, but magnetic recording means such as a hard disk drive or optical recording means using a DVD or the like may be employed.

  As shown in FIG. 2, the detection unit 6 includes two angular velocity sensors that detect the speeds around the X axis and the Y axis that are perpendicular to the Z axis that is the optical axis of the signal processing device 1. is there. By the way, camera shake at the time of shooting with a camera may also move in each direction of the X direction, Y direction, and Z direction, and rotate around the Z axis, but the Y axis is the most affected by each variation. Rotation around the X axis. These two variations are only slightly varied, and the captured image is greatly blurred. For this reason, in this embodiment, only two angular velocity sensors around the X axis and the Y axis in FIG. 2 are arranged. Therefore, the shake given to the camera is output as data of changes with time in coordinates on the XY plane. However, for the sake of completeness, an angular velocity sensor around the Z axis may be further added, or a sensor for detecting movement in the X direction or the Y direction may be added. The sensor used may be an angular acceleration sensor instead of an angular velocity sensor.

  The factor information storage unit 7 is a recording unit that stores change factor information such as known deterioration factor information, such as a transfer function calculated based on aberrations of the optical system and / or detected vibrations. The transfer function recorded in the factor information storage unit 7 includes data on the time-dependent change of coordinates on the XY plane described above. For example, the original image taken immediately after the calculation (changes such as deterioration have occurred) It is used by the processing unit 4 in the restoration process from an image) to an original image (an image before change, an image that should have been taken, or an approximate image thereof).

Here, the time-dependent change of the coordinate data on the stored XY plane is the information on the locus of blur as represented by the XY plane shown in FIG. 3 and the period of time at each position on the locus. Contains information on whether or not they stayed. The start point A (X ₁ , Y ₁ ) on the XY plane shown in FIG. 3 is the shooting start position, and the end point B (X _N , Y _N ) of the trajectory is the shooting end position.

  Degradation of an image due to blurring is a phenomenon in which light energy is not concentrated on one point but is scattered on a locus AB shown in FIG. Therefore, concentrating the dispersed light energy at one point restores the original image to the original image. One point for concentrating the light energy can be freely determined. For example, it can be determined as a point on point A, point B, point AB in FIG. 3 or a point off the point AB.

Here, the point where the dispersed light energy is concentrated is referred to as “origin position”, and the origin position is represented by a point 0 coordinate (0x, 0y) on the XY plane shown in FIG. The point spread function (PSF), which is the transfer function described above, is represented by G (Xn, Yn). This indicates “weight” which is information on how long the position (Xn, Yn) has remained. Note that (Xn, Yn) are coordinates on the XY plane shown in FIG.

Further, the movement energy for concentrating the dispersed light energy at the point 0 which is the origin position is represented by E (0x, 0y). Then, the movement energy that concentrates the dispersed energy at the origin position (0x, 0y) is the product of the movement distance and the weight, and the following equation (2) (n = 1, 2,... N: N is dispersed. The number of expanded areas).

  And the origin position (0x, 0y) which makes the movement energy E (0x, 0y) the minimum value is set. This setting is performed by the processing unit 4.

  Here, when the restoration process of the original image to the original image is performed, when the imaging power is turned off, when the processing unit 4 is not operating, when the operating rate of the processing unit 4 is low, etc. It can be a time delayed from the time when the original image was taken. In that case, the original image data stored in the recording unit 5 and the change factor information such as a transfer function for the original image stored in the factor information storage unit 7 are associated with each other for a long time. It is preserved over. As described above, the advantage of delaying the time for executing the restoration process of the original image from the time of shooting the original image is that the burden on the processing unit 4 at the time of shooting involving various processes can be reduced.

  Next, an outline of an image restoration processing method (restoration means) of the processing unit 4 of the signal processing apparatus 1 according to the present embodiment configured as described above will be described with reference to FIGS. 4, 5, and 6. .

In FIG. 4, “I ₀ ” is an arbitrary initial image and is image data stored in advance in the recording unit of the processing unit 4. “I ₀ ′” indicates data of a degraded image of I ₀ of the initial image data, and is comparison data for comparison. “G” is data of change factor information (= deterioration factor information (transfer function)) detected by the detection unit 6 and is stored in the recording unit of the processing unit 4. “Img ′” is data of the original image.

“Δ” is difference data between the original image data Img ′ and the comparison data I ₀ ′. “K” is an allocation ratio based on the data of the change factor information. “I _{0 + n} ” is restored image data (restored data) newly generated by allocating the difference data δ to the initial image data I ₀ based on the data G of the change factor information. “Img” is data of the original image. Here, the relationship between Img and Img ′ is represented by the following equation (3).
Img ′ = Img * G (3)
Here, “*” is an operator representing a superposition integral.

The difference data δ may be a simple difference between corresponding pixels, but in general, the difference data δ differs depending on the data G of the change factor information, and is expressed by the following equation (4).
δ = f (Img ′, Img, G) (4)

The processing routine of the processing unit 4 first determines the origin position where the dispersed light energy is concentrated (step S100). Then, prepare the arbitrary image data _{I 0} (step S101). As the initial image data I ₀ , the deteriorated original image data Img ′ may be used, and any image data such as black solid, white solid, gray solid, or checkered pattern may be used. good. In step S102, data I ₀ of an arbitrary image that is an initial image is inserted instead of Img in the expression (3), and comparison data I ₀ ′ that is a degraded image is obtained. Next, the original image data Img ′ and the comparison data I ₀ ′ are compared to calculate difference data δ (step S103).

Then, it is determined whether or not each absolute value of the difference data δ is less than a predetermined value (step S104). If the difference data δ is greater than or equal to a predetermined value in step S104, a process of generating new restored image data (= restored data) is performed in step S105. That is, the individual difference data δ from which individual signal elements are obtained is distributed to arbitrary image data I ₀ based on the data G of the change factor information, and new restored data I _{0 + n} is generated.

Thereafter, steps S102 to S105 in FIG. 4 are repeated. In step S104, when each absolute value of the difference data δ of each pixel becomes less than a predetermined value, the iterative process is terminated. Then, the restored data I _{0 + n} at the time when the repetitive processing is completed is estimated as the original image data Img. That is, when the absolute value of each difference data δ of each pixel becomes smaller than a predetermined value, the restored data I _{0 + n that} is the basis of the comparison data I _{0 + n} ′ is very approximate to the original image data Img. Therefore, the restored data I _{0 + n} is estimated as the original image data Img. The initial image data I ₀ and the change factor information data G may be recorded in the recording unit 5 and transferred to the processing unit 4 as necessary.

  The concept of the above-described iterative processing method (restoring means) is summarized as follows. That is, in this processing method, the processing solution is not solved as an inverse problem, but is solved as an optimization problem for obtaining a rational solution. When solving as an inverse problem, it is theoretically possible, but it is difficult as a real problem.

In the case of solving as an optimization problem, the present embodiment assumes the following conditions.
That is,
(1) The output corresponding to the input is uniquely determined.
(2) If the output is the same, the input is the same.
(3) Update the input and converge the solution so that the output is the same.

In other words, as shown in FIGS. 5A and 5B, if comparison data I ₀ ′ (I _{0 + n} ′) that is approximate to the original image data Img ′ can be generated, the original data of the generation is generated. The initial image data I ₀ or the restored data I _{0 + n} is approximate to the original image data Img.

  In this embodiment, the sampling rate of the angular velocity detection sensor is set within 60 Hz to 240 Hz. However, the angular velocity may be detected every 5 μsec so that a high frequency can be detected. In addition, in this embodiment, the value serving as a determination criterion for the difference data δ is “6” when each data is represented by 8 bits (0 to 255). That is, when it is less than 6, that is, 5 or less, the processing is finished. In addition, the raw shake data detected by the angular velocity detection sensor does not correspond to the actual shake when the sensor itself is insufficiently calibrated. Therefore, in order to cope with actual blurring, when the sensor is not calibrated, correction is required to multiply the raw data detected by the sensor by a predetermined magnification.

  Next, details of the camera shake restoration processing method (repetitive processing (restoration means) of steps S102, S103, S104, and S105) shown in FIG. 4 will be described in detail with reference to FIGS. 6, 7, 8, 9, 10, and FIG. 11, FIG. 12 and FIG. 13 will be described.

(Image restoration algorithm)
When there is no camera shake, the light energy corresponding to a given pixel is concentrated on that pixel during the exposure time. In addition, when there is a camera shake, the light energy is distributed to the blurred pixels during the exposure time. Further, if the blur during the exposure time is known, the manner in which the energy is dispersed during the exposure time can be understood, so that it is possible to create a blur-free image from the blurred image.

  Hereinafter, for the sake of simplicity, the description will be made in one horizontal dimension. Let the pixels be S-1, S, S + 1, S + 2, S + 3,... In order from the left, and pay attention to a certain pixel S. When there is no blur, the energy during the exposure time is concentrated on the pixel, so the energy concentration is “1.0”. This state is shown in FIG. The imaging results at this time are shown in the table of FIG. What is shown in FIG. 7 is the correct image data Img when no deterioration occurs. Each data is represented by 8-bit (0 to 255) data.

  Assume that there is blurring during the exposure time, 50% of the exposure time is blurred to the Sth pixel, 30% of time to the S + 1th pixel, and 20% of time to the S + 2th pixel. The way of energy dispersion is as shown in the table shown in FIG. This becomes the data G of the change factor information. The value of “N” in the above equation (1) is “3”, and the sum of 50%, 30%, and 20% as “weight” is “1”. Therefore, this change factor information G (here, G (Xn) is considered in a one-dimensional manner) satisfies the above-described equation (1).

Based on this FIG. 8 and Formula (2), the movement energy E (0x, 0y) is calculated. Here, since it is considered in one horizontal dimension, the movement energy is E (0x). Also, the movement distance is calculated by assuming that the movement distance for one pixel is “1”. Then, the movement energy in the case where the dispersed light energy is concentrated on the pixel “S” is obtained by calculating E (0x) as follows.
(1 × 0) + (0 × 0.5) + (1 × 0.3) + (2 × 0.2) = 0.7

Similarly, E (0x) is calculated and calculated as follows when the dispersed light energy is concentrated on the pixel “S + 1”.
(1 × 0.5) + (0 × 0.3) + (1 × 0.2) = 0.7

Similarly, E (0x) is calculated and obtained as follows when the dispersed light energy is concentrated on the pixel “S + 2”.
(2 × 0.5) + (1 × 0.3) + (0 × 0.2) = 1.3

  From the above results, in the case of FIG. 8, the movement energy can be set to the minimum value “0.7” by concentrating the dispersed light energy on the pixel “S” or “S1”. Further, in the case of “S = 0.45”, “S + 1 = 0.3”, and “S + 2 = 0.25” instead of FIG. 8, the sum of the kinetic energy to the pixel “S + 1” is the smallest. That is, the movement to the pixel “S” is “0.8”, the movement to the pixel “S + 1” is “0.7”, and the movement to the pixel “S + 2” is “1.2”. Hereinafter, the details of the iterative process when the dispersed light energy is concentrated on the position where the moving energy is the smallest, that is, on the pixel “S” in the example of FIG. 8 described above will be described.

  Blur is uniform for all pixels and is understood as a linear problem. If there is no upper blur (vertical blur), the blur situation is as shown in the table of FIG. The data shown as “blurred image” in FIG. 9 becomes the degraded original image data Img ′. Specifically, for example, “120” of the pixel “S-3” is in accordance with the distribution ratio of “0.5”, “0.3”, “0.2” of the data G of the change factor information that is blur information, Dispersed in such a manner that “60” is distributed to the “S-3” pixel, “36” is distributed to the “S-2” pixel, and “24” is distributed to the “S-1” pixel. Similarly, “60” which is the pixel data of “S-2” is distributed as “30” in “S-2”, “18” in “S-1”, and “12” in “S”. The original image data Img is calculated from the deteriorated original image data Img ′ and the change factor information data G shown in FIG. The above processing is processing for moving part or all of the signal element data.

The arbitrary image data I ₀ shown in step S101, can be adopted also What, When this description, using the original image data Img '. That is, the process starts with I ₀ = Img ′. In the table of FIG. 10, “input” corresponds to the initial image data I ₀ . This data I _0, that is, Img ′ is superposed and integrated with the data G of the change factor information in step S102. That is, for example, “60” of the “S-3” pixel of the initial image data I ₀ is “30” for the S-3 pixel, “18” for the “S-2” pixel, “12” is assigned to each pixel of “−1”. The other pixels are similarly distributed, and comparison data I ₀ ′ shown as “output I ₀ ′” is generated. Therefore, the difference data δ in step S103 is as shown in the bottom column of FIG.

As shown in FIG. 11, the distribution of the difference data δ is, for example, a distribution ratio of 0.5 to the pixel data “30” of “S-3” (= the pixel of “S-3”). “15” multiplied by “S-3” is distributed to the pixel of “S-3”, and the data “15” of the pixel of “S-2” is allocated to the pixel of “S-2”. “4.5” multiplied by 0.3 is allocated, and further, the data “9.2” of the pixel “S-1” is allocated to the pixel “S-1” by the distribution ratio. “1.84” multiplied by a certain 0.2 is allocated. The total amount allocated to the pixels of “S-3” is “21.34”, and this value is added to the initial image data I ₀ (in this case, the original image data Img ′) in FIG. The restored data I _{0 + 1 at 4} is calculated. In this example, “81.34” is obtained as shown in FIG. In this way, the difference data δ is distributed to the arbitrary image data I ₀ using the change factor information data G to generate the restored data I _{0 + n} shown as “next input” in FIG. . In this case, since this is the first time, it is represented as I _{0 + 1 in} FIG.

As shown in FIG. 12, the restored data I _{0 + 1} (Ia ′) becomes the input image data (= initial image data I ₀ ) in step S102, step S102 is executed, and the process proceeds to step S103. The difference data δ is obtained. The size of the difference data δ is determined in step S104. If the difference data δ is larger than the predetermined value, the new difference data δ is distributed to the previous restored data I _{0 + 1} in step S105 to generate new restored data I _{0 + 2} (FIG. 13).

Thereafter, restoration by the data _{I 0 + 2} with performing step S102, the restored data _{I 0 + 2} new comparison data from the _{I 0 + 2} ^'is produced. As described above, after steps S102 and S103 are executed, the process proceeds to step S104, and the process proceeds to step S105 based on the determination. Such a process is repeated.

  As described above, by repeating Step S102 to Step S105, the difference data δ gradually decreases. When the difference data δ becomes smaller than a predetermined value, the original image data Img without blur is obtained. The original image data Img is obtained by returning (moving) the dispersed energy to the pixel “S” in the example shown in FIG.

  In the camera shake restoration processing method shown in FIG. 4 described above (repetitive processing of steps S102, S103, S104, and S105), the processing performed by the processing unit 4 is configured by software. You may comprise with the hardware which consists of components which performed the process of the part in a shared manner. Further, the change factor information data G includes not only the deterioration factor information data but also information for simply changing an image and information for improving an image contrary to deterioration.

  Further, when the number of processing iterations is set automatically or fixedly on the signal processing device 1 side, the set number of times may be changed by the data G of the change factor information. For example, when the data of a certain pixel is distributed over many pixels due to blurring, the number of iterations may be increased, and when the variance is small, the number of iterations may be decreased.

  Furthermore, during the iterative process, if the difference data δ diverges or the energy of the image data after the energy has moved does not decrease but increases, the process may be stopped. For example, it is possible to adopt a method of determining whether or not the light is diverging by observing the average value of the difference data δ and determining that the light is diverging if the average value is larger than the previous value. In addition, during an iterative process, if an input is to be changed to an abnormal value, the process may be stopped. For example, in the case of 8 bits, if the value to be changed is a value exceeding 255, the processing is stopped. Further, during an iterative process, when an input that is new data is to be changed to an abnormal value, the value may not be used but may be set to a normal value. For example, when a value exceeding 255 within the 8-bit range of 0 to 255 is used as input data, it is processed as a maximum value of 255.

  In addition, when generating restoration data to be an output image, depending on the data G of the change factor information, there may be data that goes out of the area of the image to be restored. In such a case, data that protrudes outside the area is input to the opposite side. Also, if there is data that should come from outside the area, it is preferable to bring that data from the opposite side. For example, when data allocated to a lower pixel is generated from the data of the pixel XN1 positioned at the bottom in the area, the position is outside the area. Therefore, the data is processed to be allocated to the pixel X11 located at the top right above the pixel XN1. Similarly, the pixel N2 adjacent to the pixel XN1 is assigned to the topmost pixel X12 (= next to the pixel X11) directly above.

  When the dispersed light energy is 0.85 for the pixel “S”, which is the modified example of FIG. 8, 0.3 for the pixel “S + 1”, and 0.25 for the pixel “S + 2”, the pixels “S”, “S + 1”, The process was repeatedly performed as shown in FIG. 4 by concentrating each on “S + 2” to obtain the origin position. The restored image in each case was visually observed to determine the presence or absence of ringing. Further, “N = 3”, which is the calculation range of Equation (2), is slightly expanded, and the dispersed light energy is concentrated on the pixels “S−1” and “S + 3” in FIG. The presence / absence of ringing of the restored image in the case where the repetitive processing shown was performed was similarly determined. Table 1 shows the determination results.

Note that the movement energy in the case where the dispersed light energy is concentrated on the pixel “S−1” is calculated and obtained as follows in substantially the same manner as in the case of the pixels “S”, “S + 1”, and “S + 2”.
(0 × 0) + (1 × 0.45) + (2 × 0.3) + (3 × 0.25) + (4 × 0) = 1.80
Similarly, the movement energy when the dispersed light energy is concentrated on the pixel “S + 3” is calculated and obtained as follows.
(4 × 0) + (3 × 0.45) + (2 × 0.3) + (1 × 0.25) + (0 × 0) = 2.20

  From the results in Table 1 and many other examples, it was found that ringing was observed when the kinetic energy value exceeded a predetermined value, and ringing could be suppressed by keeping the kinetic energy value within a predetermined value. That is, when the minimum value of the total kinetic energy derived from Equation (2) is Min, ringing occurs if the kinetic energy exceeds Min × 1.2 or less. Was able to be suppressed considerably compared with the past.

  The signal processing device 1 and the examples in the present embodiment have been described above, but various modifications can be made without departing from the gist of the present invention. For example, in the present embodiment, the origin position (0x, 0y) that minimizes the sum of the kinetic energy E (0x, 0y) is set, but the minimum value of the kinetic energy E (0x, 0y) is exceeded. An origin position (0x, 0y) that is equal to or less than a predetermined value may be set. In the case of “S = 0.45”, “S + 1 = 0.3”, and “S + 2 = 0.25”, which are the modified examples of FIG. 8, the pixel “S” is not the pixel “S + 1” having the smallest kinetic energy, but the origin position (0x, 0y) may be used. The kinetic energy at this time is 0.8 for the pixel “S”, 0.7 for the pixel “S + 1”, and 1.2 for the pixel “S + 2”. The pixel “S” has a value smaller than 0.84, which is a value obtained by multiplying the minimum value 0.7 by 1.2.

  Further, in the present embodiment, the above-described repetitive processing is adopted as the restoration means, but other restoration means may be adopted as long as it is a means for moving part or all of the signal element data.

  In the present embodiment, the origin position is any position on the XY plane in FIG. However, the origin position may be determined within a range on the trajectory AB in FIG. That is, for example, any one of the pixels “S”, “S + 1”, and “S + 2” in FIG. 8 that is within the range of the AB trajectory in FIG. 3 or the pixel “S−1” or “S + 3” is the origin position. It is preferable to do so if the total amount of kinetic energy is reduced. Also, the blur locus represented by the XY plane in FIG. 3 projected on the X-axis or Y-axis is defined as the blur locus, and the minimum value of the kinetic energy on the locus or the value as in the above-described embodiment. You may make it obtain | require the position which becomes.

  In the iterative process according to the present embodiment, the processing unit 4 determines whether or not the absolute values of the difference data δ for each of the plurality of pixels constituting the image are all predetermined values in the determination in step S104 in FIG. Judging. However, the comparison target with the predetermined value may be the difference data for each of the plurality of pixels constituting the image, and it may be determined whether or not the repeated processing is stopped for each pixel. The comparison target with the predetermined value may be the sum of the difference data δ of each pixel, the sum of the absolute values of the difference data δ of each pixel, or two or more of the above four. For example, it is determined whether the value farthest from zero in the difference data δ for each pixel and the sum of the difference data δ for each pixel satisfy different criteria. May be. As described above, by appropriately selecting a value to be compared with the predetermined value, appropriate processing can be performed according to the type of original image, the state of change, or the status of restoration processing.

  In the above-described embodiment, the restoration target is image data. However, these restoration processing concepts and techniques can be applied to any digital data restoration processing. For example, it can be applied to restoration of digital audio data. As a result of the application, it is possible to suppress the occurrence of some inaccurate audio data, such as ringing, and even if the data of the change factor information is inaccurate, it is possible to perform restoration processing that can obtain reasonable results It becomes.

  In the above-described embodiment, the signal processing device 1 is a consumer camera. However, the signal processing device 1 prints image data captured by a digital camera or the like after executing the processing shown in FIG. It may be a printer device. Further, the signal processing apparatus 1 may be a computer installed with software for operating the printer device while performing the processing shown in FIG. 4, or a computer installed with software for executing the processing shown in FIG. good.

  Moreover, each processing method mentioned above may be programmed. Alternatively, the program may be stored in a storage medium, such as a CD, DVD, or USB memory, and read by a computer. In this case, the signal processing device 1 may be a program stored in the storage medium placed in an external server of the signal processing device 1 and downloaded and used as necessary. In this case, the signal processing apparatus 1 has communication means for downloading the program in the storage medium.

It is a block diagram which shows the main structures of the signal processing apparatus which concerns on embodiment of this invention. It is an external appearance perspective view which shows the outline | summary of the signal processing apparatus shown in FIG. 1, and is a figure for demonstrating the arrangement position of an angular velocity sensor. It is a figure which shows the example of the blur handled with the signal processing apparatus shown in FIG. 1, and is a figure which shows the locus | trajectory of the blur represented by XY plane. FIG. 2 is a processing flowchart for explaining a processing routine related to an image restoration processing method (repetitive processing) performed by a processing unit of the signal processing device shown in FIG. 1. It is a figure for demonstrating the concept of the processing method shown in FIG. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 3 using camera shake as an example, and is a table showing energy concentration when there is no camera shake. It is a figure for demonstrating concretely the processing method shown in FIG. 4 as an example of camera shake, and is a figure which shows image data when there is no camera shake. It is a figure for demonstrating concretely the processing method shown in FIG. 4 as an example of camera shake, and is a figure which shows dispersion | distribution of energy when camera shake occurs. FIG. 5 is a diagram for specifically explaining the processing method illustrated in FIG. 4 using camera shake as an example, and is a diagram for illustrating a situation in which comparison data is generated from an arbitrary image. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 4 using camera shake as an example, in which comparison data and a blurred original image to be processed are compared to generate difference data It is a figure for demonstrating. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 4 using camera shake as an example, and a diagram for explaining a situation in which restored data is generated by allocating difference data and adding it to an arbitrary image. is there. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 4 using camera shake as an example. New comparison data is generated from the generated restoration data, and the blurred original image to be processed It is a figure for demonstrating the condition which produces | generates the data of a difference by comparing. FIG. 4 is a diagram for specifically explaining the processing method shown in FIG. 4 using camera shake as an example, and a diagram for explaining a situation in which newly generated difference data is allocated and new restored data is generated. It is.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Signal processing apparatus 4 Processing part Io Initial image data (arbitrary signal data)
Io 'Comparison data G Change factor information data Img' Original image data (original signal data)
I _{0 + n} restored data Img Original image (original signal)
δ Difference data

Claims (3)

From the original signal data that has undergone changes such as degradation, it has a processing unit that restores the signal before the change or the signal that should have been originally acquired or their approximate signal (hereinafter referred to as the original signal),
The processing unit includes a restoration unit that generates restoration data to be an original signal using data of change factor information that causes a signal change,
The restoring means is a means for generating the original signal by moving some or all of the data of the signal components constituting the original signal data and the restored data from the original signal data,
The signal processing is characterized in that the processing unit sets the origin position of the change factor information to a position where the total amount of kinetic energy required to move the data of each of the signal elements is a minimum value, and performs a restoration process. apparatus. From the original signal data that has undergone changes such as degradation, it has a processing unit that restores the signal before the change or the signal that should have been originally acquired or their approximate signal (hereinafter referred to as the original signal),
The processing unit includes a restoration unit that generates restoration data to be an original signal using data of change factor information that causes a signal change,
The restoring means is a means for generating the original signal by moving some or all of the data of the signal components constituting the original signal data and the restored data from the original signal data,
When the origin of the change factor information is Min, and the minimum value of the total kinetic energy required to move the data of each of the signal elements is Min, the processing energy exceeds Min, and Min × 1. A signal processing apparatus that performs restoration processing by setting the position to be 2 or less.   The restoration means uses the data of the change factor information that causes the signal to be changed by the processing unit, generates comparison data from arbitrary signal data, and compares the data with the original signal data to be processed. And the difference data obtained is distributed to the arbitrary signal data using the data of the change factor information in which the origin position is set, and the restoration data is generated. 3. The signal processing apparatus according to claim 1, wherein the signal processing apparatus is a means for performing repetitive processing by using the restored data instead of the arbitrary signal data and repeating the same processing.

Images