JP7427239B2 - Signal processing device, signal processing method and program - Google Patents

Description

The present invention relates to a signal processing device, a signal processing method, and a program. The present invention estimates an estimated value of a signal to be analyzed in a reference interval using, for example, a characteristic parameter obtained by performing frequency analysis on a signal to be analyzed in a peripheral interval set around a reference interval, This invention relates to technology for reducing noise added to signals to be analyzed.

Various methods have been proposed for reducing noise added to analysis target signals such as images. For example, the image processing method described in Patent Document 1 includes a step of convolving an ideal image using PSF to generate a predicted image, a step of performing differential processing on an original image from the predicted image to generate an error image, and a step of generating an error image by performing differential processing on an original image from the predicted image. A step of performing a deconvolution operation using the PSF multiplied by a coefficient, and performing a difference process on the image after the convolution operation from the ideal image to generate a difference image, and repeating the above steps multiple times with the difference image as a new ideal image to generate a noise image. After obtaining the modified original image, the modified original image is obtained by subtracting the noise image from the original image.

Furthermore, in Non-Patent Document 1, a plurality of similar blocks similar to a partial image in a reference block that is a part of an image to be analyzed are searched for by block matching, and a portion of each of the plurality of similar blocks is Describes a method of stacking images to form a three-dimensional group, performing Wiener filtering in the three-dimensional frequency domain, and averaging filtered similar blocks that are inversely transformed to the spatial domain to determine the estimated value of the reference block. has been done.

Japanese Patent Application Publication No. 2019-144808 Patent No. 5590547

Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian, “Image denoising by sparse 3D transform-domain coll aborative filtering”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 8, pp. 1-16, AUGUST 2007

Conventional methods, including the above-mentioned method, use similar blocks that are similar to the reference block among blocks that exist in different parts from the reference block. In searching for similar blocks, an index value that indicates the degree of similarity between all signal values in a reference block and all signal values in a candidate block is usually used. As such an index value, for example, an L2 norm based on the luminance value of each pixel may be used. However, the image to be analyzed does not necessarily include similar blocks that match the reference block even in details such as edge areas where brightness changes significantly depending on position. On the other hand, when similar blocks with different edge distributions are used to suppress noise, spatial changes in brightness are averaged between reference blocks, resulting in loss of detailed information and a decrease in image quality. Something happened.

The present invention has been made in view of the above points, and an object of the present invention is to suppress noise while suppressing deterioration in quality of a signal to be analyzed.

The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is to calculate a characteristic parameter indicating the characteristic of the signal to be analyzed from the signal to be analyzed in each of a plurality of first sections. an analysis unit; an estimation unit that generates an estimated signal for each section including an estimated value of the signal to be analyzed in a second section based on the characteristic parameter for each of the first sections; and an estimated signal for each section of the first section. a processing unit that averages the estimated signal to generate an estimated signal , the analysis unit calculates the amplitude and phase of each frequency component as the characteristic parameters, and the estimation unit calculates the first interval for each frequency component. A correction value of the phase is calculated based on the displacement from each of the amplitudes and the correction values of the phase to the second section, a reconstructed value in the second section is calculated based on the amplitude and the corrected value of the phase, and the reconstructed value is This is a signal processing device that generates the interval-specific estimation signal by synthesizing the frequency-specific estimation signals including the frequency-specific estimation signals .

According to this embodiment, it is possible to suppress noise while suppressing deterioration in the quality of the signal to be analyzed.

1 is a schematic block diagram showing an example of a functional configuration of a signal processing device according to an embodiment. FIG. 1 is a schematic block diagram showing an example of a hardware configuration of a signal processing device according to an embodiment. FIG. FIG. 2 is an explanatory diagram for explaining an example of a method for generating a section-by-section estimation signal according to the present embodiment. FIG. 3 is a diagram illustrating an example of reconstructing an estimated signal. FIG. 7 is an explanatory diagram for explaining another example of the method for generating a section-by-section estimated signal according to the present embodiment. FIG. 7 is a diagram showing another example of reconstruction of an estimated signal. FIG. 3 is a diagram showing an example of an image to be analyzed. FIG. 7 is a diagram showing another example of an image to be analyzed. 7 is a flowchart illustrating an example of noise suppression processing according to the present embodiment. FIG. 3 is a diagram showing an example of an original image and a reconstructed image. FIG. 3 is a diagram showing an example of a standard image to be analyzed. Examples of an original image, a noise-added image, and a reconstructed image based on a standard image are shown. 7 is a table showing a first example of estimation accuracy of a reconstructed image. It is a table showing a second example of estimation accuracy of reconstructed images. A first example of an original image, a noise-added image, and a reconstructed image based on a high-resolution image is shown. A second example of an original image, a noise-added image, and a reconstructed image based on a high-resolution image is shown. It is a table showing an example of estimation accuracy of high-resolution images.

(Configuration of signal processing device)
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing an example of the functional configuration of a signal processing device 10 according to this embodiment.
The signal processing device 10 acquires a signal to be analyzed, and calculates, for each of the plurality of first sections, a characteristic parameter indicating the characteristic of a series of signal values forming the signal to be analyzed within that section. The signal processing device 10 calculates an estimated value of the signal to be analyzed in a second section that is separate from the plurality of first sections, based on the characteristic parameters calculated for each of the plurality of first sections. A series including the estimated values is generated as a section-specific estimated signal. Then, the signal processing device 10 averages the estimated values included in the section-by-section estimated signals generated for each of the plurality of first sections, and generates a sequence whose signal value is the averaged estimated value as the estimated signal.

The signal to be analyzed is not limited to a two-dimensional signal in which multiple signal values are arranged in a two-dimensional space, such as an image signal representing a plane image of one frame, but can also be a one-dimensional signal or a high-dimensional signal of three or more dimensions. There may be. Furthermore, the types of signals to be analyzed are not limited to image data representing images, but include audio data representing sounds, acceleration signals representing time changes in acceleration, biological signals representing the activity status of the heart and other biological organs, and so on. Any series may be used as long as it shows a spatial or temporal change in a physical quantity. However, in the following description, the case where the signal to be analyzed is an image signal representing one frame of a plane image will be mainly exemplified.

The signal processing device 10 includes a control section 120 and a storage section 140. The control unit 120 controls various functions that the signal processing device 10 has. The control unit 120 may be a dedicated member, or may include one or more processors 102 (described later) such as a CPU (Central Processing Unit). The processor 102 provides the function of the control unit 120 by executing processes instructed by various instructions written in a predetermined program stored in the storage unit 140 in advance. In the following description, executing a process instructed by instructions written in various programs may be referred to as executing a program, executing a program, or the like. Note that the control section 120 is connected to the storage section 140 via a data bus so as to be able to input and output various types of data to and from each other.

The storage unit 140 is a storage medium such as an HDD (Hard Disk Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory) 104 (described later), and a RAM (Random Access Memory) 106 (described later). It consists of a combination of some or all of the following. The storage unit 140 stores various data in addition to programs.

Next, an example of the functional configuration of the control unit 120 will be described. The control unit 120 includes a signal acquisition unit 122, a characteristic analysis unit 124, a signal estimation unit 126, and an emphasis processing unit 128. In addition, the control unit 120 performs processing to perform the functions of the signal processing device 10.

The signal acquisition unit 122 acquires an analysis target signal to be subjected to signal processing according to the present embodiment. Image data specified by an operation signal input from 108 (described later) is specified, and the specified image data is read out as a signal to be analyzed. The signal acquisition unit 122 may receive image data input via the input/output unit 112 as an analysis target signal. The signal acquisition unit 122 outputs the acquired analysis target signal to the characteristic analysis unit 124.

The characteristic analysis unit 124 identifies a reference interval to be referred to as a processing target and a plurality of surrounding intervals that are separate intervals from the reference interval, among the intervals forming the analysis target signal input from the signal acquisition unit 122. . In this application, an interval refers to a sequence that is part of the signal to be analyzed and is composed of multiple signal values, and is a time series in which signal sequences are arranged in order of time and position. It can include any spatial sequence in which signal values are arranged in order. When the signal to be analyzed is image data indicating a planar image, each section is a part of the entire area of the planar image, and each section is a region at a different position. That is, each section corresponds to an image block (also called a patch, a processing unit, etc.) in which a plurality of pixels are arranged in a two-dimensional plane. In the following description, an image block may be simply referred to as a block, and a partial image within a block corresponding to a part of a planar image or a signal indicating a partial image may be referred to as a patch. Further, a reference section and a surrounding section within a planar image may be called a reference block and a surrounding block, respectively, and a partial image within a reference block and a partial image within a surrounding block may be called a reference patch and a surrounding patch, respectively.

The reference section may be set in advance in the characteristic analysis section 124, or the characteristic analysis section 124 may set a section specified by an operation signal input from the operation input section 108 (described later) or the input/output section 112 (described later). may be specified as the reference interval. Furthermore, the characteristic analysis unit 124 identifies a section that is different from the reference section and has the same size as the reference section as a surrounding section. The plurality of surrounding sections may be set over the entire section of the analysis target signal excluding the reference section, or may be set only on some sections other than the reference section. For example, the characteristic analysis unit 124 may set a section (for example, an adjacent section) that exists within a predetermined range of time or space from the reference section as the surrounding section. When the signal to be analyzed is a planar image, the reference block is specified by position and size, and each surrounding block is specified by a different position.

The characteristic analysis unit 124 calculates a characteristic parameter indicating the characteristic of the signal to be analyzed from the signal to be analyzed in each of the plurality of surrounding sections. For example, the characteristic analysis unit 124 uses non-harmonic analysis (NHA) on surrounding patches representing partial images of each surrounding block to determine the signal value of each pixel forming the surrounding patch. A set of sequence amplitudes, phases, and frequencies is defined for each frequency component. Each frequency component corresponds to a set of two-dimensional spatial frequencies. The characteristic analysis section 124 outputs the characteristic parameters calculated for each surrounding section to the signal estimation section 126. An example of a method for calculating characteristic parameters will be described later.

The signal estimation unit 126 determines the analysis target in the reference interval based on the characteristic parameters for each surrounding interval inputted from the characteristic analysis unit 124 and the positional relationship between the surrounding interval and the reference interval. Calculate the signal estimate. The sequence of signals that are individualized and include the calculated estimated values corresponds to the section-based estimated signal. Therefore, a section-by-section estimation signal is generated for each of the plurality of surrounding sections. For example, the signal estimation unit 126 calculates a phase correction value based on the displacement of the phase calculated for each frequency component from the surrounding blocks to the reference block. The signal estimation unit 126 calculates a frequency-specific signal value as a signal value for each pixel based on the amplitude, frequency, and calculated phase correction value for each frequency component. Then, the signal estimation unit 126 adds (integrates) the calculated frequency-specific signal values between frequency components for each of the surrounding blocks to calculate a section-specific estimated signal value that forms a section-specific estimated signal. As a result, a block-based estimated patch is generated as an example of a section-based estimated signal made up of section-based estimated signal values. The signal estimator 126 outputs the section-by-section estimated signals generated for each of the surrounding sections to the emphasis processing section 128. An example of a method for generating the section-by-section estimation signal will be described later.

The emphasis processing unit 128 performs averaging processing on the section-specific estimated signals of the plurality of surrounding sections inputted from the signal estimation section 126, and generates an estimated signal that is a sequence including averaged signal values in the reference section. Generate a signal. The signal value included in the estimated signal corresponds to the estimated value of the signal to be analyzed within the reference interval. Therefore, the noise components included in the section-wise estimated signal are relatively reduced by the averaging process, and the signal components are relatively emphasized. In addition, since the components included in the section-specific estimated signals that differ for each reference section are complemented, deterioration in quality due to omission of specific components is prevented. The emphasis processing unit 128 can, for example, perform emphasis filtering, which is an example of averaging processing, on each block-specific estimated patch of the surrounding blocks to generate the estimated patch of the reference block as an estimated signal. The emphasis processing unit 128 may store the generated estimated signal in the storage unit 140, or may output it to a device separate from the signal processing device 10 using the input/output unit 112. When the estimated signal is image data, the emphasis processing section 128 may output the image data to the display section 110 (described later).

Further, the control unit 120 may change the estimated interval to an unprocessed interval and sequentially repeat a series of processes up to generating the above-mentioned estimation signal for the changed estimated interval. The control unit 120 may collectively output a plurality of estimated signals generated through repetition. An example of emphasis filtering will be described later.

Next, an example of the hardware configuration of the signal processing device 10 according to this embodiment will be described. FIG. 2 is a schematic block diagram showing an example of the hardware configuration of the signal processing device 10 according to the present embodiment. The signal processing device 10 illustrated in FIG. 2 is configured as a computer. The signal processing device 10 includes a processor 102, a ROM 104, a RAM 106, an operation input section 108, a display section 110, and an input/output section 112.

The processor 102 is capable of executing various programs such as application programs, utilities, drivers, and operating systems, and controls the operations of each component of the signal processing device 10 .
The ROM 104 can store various programs and various data. Programs and data stored in the ROM 104 can be read out under the control of the processor 102.
The RAM 106 temporarily stores various programs and data under the control of the processor 102, and reads out the stored programs and data. The storage area of the RAM 106 is used as a work area when the processor 102 executes a program.

The operation input unit 108 receives a user's operation, generates an operation signal according to the received operation, and outputs the generated operation signal to the processor 102. The operation signal indicates various information such as user instructions and commands to the signal processing device 10. The operation input unit 108 includes any type of operation device such as a keyboard, a mouse, a keypad, a push button, and a stick key. The operation input unit 108 may include a receiver that receives wireless (including infrared rays) from a physically separated operation device (for example, a remote controller).

Under the control of the processor 102, the display unit 110 displays information specified by various display data inputted from the processor 102 in a visually recognizable manner. Such display data includes image data, text data, and the like. The display unit 110 is any type of display device such as a liquid crystal display (LCD), a plasma display panel (PDP), or an organic electro-luminescence display (OLED). It consists of:

The input/output unit 112 is connected to a device separate from the signal processing device 10 by wire or wirelessly, and inputs and outputs various data. The input/output unit 112 may be connected to a communication network by wire or wirelessly, and may transmit and receive various data to and from separate devices connected to the communication network. The input/output unit 112 includes, for example, input/output devices such as an input/output interface and a communication interface.

(Method of generating estimated signal by section)
Next, an example of a method for generating an estimated signal for each section will be described. FIG. 3 is an explanatory diagram for explaining an example of a method for generating a section-by-section estimated signal according to the present embodiment. In the example shown in FIG. 3, the signal to be analyzed is a one-dimensional signal having a signal value that changes over time. In FIG. 3, the vertical and horizontal axes indicate amplitude A and time t, respectively. The surrounding section w32 and the reference section w34 are each time windows of the period T. The displacement d indicates the time from the surrounding section w32 to the reference section w34.

The process of generating the section-by-section estimated signal within the reference section from the analysis target signal within the surrounding section includes the following steps.
(Step S12) The characteristic analysis unit 124 uses the peripheral section w32 of the signal to be analyzed as an analysis window, and extracts a portion defined by the analysis window (extraction). In the example shown in FIG. 3, it is assumed that the signal value at each time t of the analysis target signal in the surrounding section w32 is expressed as Acos (2πft+φ). f and φ indicate frequency and phase, respectively.
(Step S14) The characteristic analysis unit 124 calculates characteristic parameters for each frequency component for the extracted signal (analysis). The calculated characteristic parameters are phase φ, frequency f, and amplitude A.

(Step S16) The characteristic analysis unit 124 adjusts the phase φ among the calculated characteristic parameters, and generates a frequency-specific signal in which the characteristics of the frequency component are expressed by the adjusted phase φ′, frequency f, and amplitude A between the frequency components. are combined to generate an estimated signal for each section of the reference section w34 (reconstruction).
In step S16, the characteristic analysis unit 124 adds an adjustment value fd/T obtained by multiplying the displacement d by the wave number f/T corresponding to the frequency f to the phase φ determined for the surrounding section w32, and after the adjustment, Calculate the phase φ' of In the example shown in FIG. 3, the signal value of the reconstructed section-wise estimated signal is expressed as Acos (2πft+φ+fd/T).

(How to calculate characteristic parameters)
Next, an example of a method for calculating characteristic parameters according to this embodiment will be described. The characteristic analysis unit 124 can use, for example, NHA as a method for calculating characteristic parameters indicating the characteristics of the series of signal values forming the analysis target signal in the surrounding section. NHA is a method in which the magnitude of the frequency component residual, which is the difference between the signal value s(x) constituting the signal to be analyzed and the frequency component A'cos (2πf'x+φ'), is minimized for all samples n within the analysis window. This is a method of calculating a set of frequency f', amplitude A', and phase φ' as characteristic parameters of the frequency component. Further, the characteristic analysis unit 124 uses the residual frequency component whose frequency component residual has been minimized as a new signal value, and sequentially repeats the process of calculating characteristic parameters for new separate frequency components. Thereby, a set of characteristic parameters is calculated for each frequency component. The number of repetitions of the process of calculating characteristic parameters for each frequency component may be a preset integer of 1 or more (usually 2 or more). The number of repetitions corresponds to the number of frequency components for which characteristic parameters are calculated. In addition, the process of calculating the characteristic parameters is repeated until the magnitude of the residual error of the frequency component or the ratio of the magnitude of the residual error of the frequency component to the magnitude of the signal to be analyzed becomes equal to or less than a preset lower limit, without specifying the number of repetitions. It's okay. According to NHA, since the frequencies of multiple frequency components are not necessarily distributed at equal intervals, the number of characteristic parameters that indicate the characteristics of the signal to be analyzed can be reduced compared to existing methods such as discrete Fourier transform. Can be done. Furthermore, the influence of the analysis window on analysis accuracy is small, and the frequency dependence of analysis accuracy is suppressed.

FIG. 4 is a diagram showing an example of reconstructing an estimated signal. FIG. 4 shows how to analyze characteristic parameters using the characteristic parameters calculated in the analysis interval using the original signal in the analysis interval as the analysis target signal for each of Fast Fourier Transform (FFT) and NHA. The estimated signals reconstructed in each of two adjacent sections adjacent to the analysis section are illustrated. The first, second, and third stages in FIG. 4 show the original signal, the estimated signal based on FFT, and the estimated signal based on NHA, respectively. However, the original signal is shown as the signal within the analysis section in the second and third stages.

FFT is a form of discrete Fourier transform, and is a method of calculating a predetermined amplitude and phase for each frequency component as characteristic parameters. The characteristic analysis unit 124 can reconstruct the estimated signal in the adjacent section using the obtained amplitude and phase. As shown in the second row of FIG. 4, significant discontinuities occur between the signal values of the original signal at times at both ends of the analysis interval and the signal values of the estimated signal at adjacent times.

On the other hand, for the estimated signal reconstructed based on the characteristic parameters obtained using NHA, there is a difference between the signal value of the original signal at both ends of the analysis interval and the signal value of the estimated signal at adjacent times. No discontinuity occurs (see paragraph 3). This shows that the characteristic parameters calculated using NHA can reduce the influence of the analysis window and improve the estimation accuracy of the signal to be analyzed. In other words, according to NHA, it is possible to express the characteristics of a signal to be analyzed in an analysis section with high precision and a small number of characteristic parameters, and it is also confirmed that the signal to be analyzed in a section separate from the analysis section can be estimated with high precision.

In NHA, when using a one-dimensional signal as an analysis target signal, the characteristic analysis unit 124 calculates an amplitude A' that minimizes the cost function F (A', f _x ', φ') shown on the left side of equation (1), Search for frequency f _x ' and phase φ'. The cost function F(A', f _x ', φ') is an example of an index value of the magnitude of the frequency component residual s(x _n )−A'cos(2πf'x _n +φ').

However, in Equation (1), N is an integer of 2 or more indicating the number of samples of the signal value s(x _n ) within the analysis window. x _n indicates the coordinates of the n-th sample in the one-dimensional section forming the analysis window. If interval means space, coordinates means position within space. If interval means time, coordinates means time within time. The coordinates of individual samples do not necessarily have to be distributed at equal intervals in the analysis window, and the intervals may not be constant values but may be set randomly.
When using a two-dimensional signal as the signal to be analyzed, the characteristic analysis unit 124 calculates the amplitude A that minimizes the cost function F (A', f _x ', f _y ', φ') shown in equation (2). ', two-dimensional frequencies f _x ', f _y ', and phase φ'.

However, in Equation (2), N _x and N _y are integers of 2 or more indicating the number of samples in each of the dimensions x and y of the signal value s (n _x , _ny ) within the analysis window, respectively. x _nx and y _ny indicate the coordinates of the n _x and n _y samples of the dimensions x and y, respectively, in the two-dimensional interval forming the analysis window.
In this application, minimization is not limited to calculating or finding the absolute minimum value, but also includes searching as much as possible for a value (for example, a characteristic parameter) that gives the minimum value, and by that process, the index value may increase temporarily. As a method of minimization, for example, the steepest descent method can be used. The steepest descent method calculates the partial differentiation of the cost function with respect to each element value of the characteristic parameter (in this embodiment, the frequency f, amplitude A, and phase φ of each dimension correspond to each element value) at that point in time (current point). This is a method of recursively repeating the process of calculating the element value at the next point in time by subtracting an updated value obtained by multiplying the value by a predetermined coefficient. The process is repeated, for example, until the absolute value of the difference between the cost function based on the characteristic parameters at the current point in time and the cost function based on the characteristic parameters at the next point in time, that is, the amount of change in the cost function, becomes equal to or less than a predetermined change amount threshold. do it.
In particular, when the two-dimensional signal to be analyzed is image data representing one frame of a plane image, the characteristic analysis unit 124 uses the cost function F(A What is necessary is to search for the amplitude A', the two-dimensional frequencies f _x ', f _y ', and the phase φ' that minimize _the amplitude A', f x ', f _y ', φ').

However, in the example of equation (3), the pixels are arranged at constant intervals (pixel pitches) 1/f _sx and 1/f _sy in the x direction (horizontal direction) and y direction (vertical direction) of the analysis target area, respectively. It is assumed that there is. Therefore, the frequencies f _x ′ and f _y ′ in the x and y directions are normalized by the sampling frequencies f _sx and f _sy corresponding to the intervals, respectively. Here, a pixel in which both n _x and n _y are 0 indicates a pixel located at the origin of the analysis target area. Therefore, by setting the area to be analyzed as a surrounding block that is a part of an image of one frame, the characteristic analysis unit 124 calculates the amplitude A′, frequency f _x ′, f _y ′, and phase φ of the partial image within the surrounding block. ' can be calculated.

Note that NHA is described in detail in Patent Document 2. The signal to be analyzed is not limited to one-dimensional signals or two-dimensional signals, but NHA is also applicable to high-dimensional signals of three or more dimensions. In that case as well, amplitude, frequency, and phase are calculated as characteristic parameters of each frequency component. The number of dimensions of the calculated frequency corresponds to the number of dimensions of the signal to be analyzed.

Next, another example of a method for generating a section-based estimated signal will be described. FIG. 5 is an explanatory diagram for explaining an example of a method for generating a section-by-section estimated signal according to the present embodiment. FIG. 5 illustrates a process of generating a block-specific estimated patch, which is a partial image of a reference block, as a section-specific estimation signal using surrounding blocks of image data representing a plane image of one frame as an analysis section. The reference block and the surrounding block illustrated in FIG. 5 are both rectangular areas having a width of X in the horizontal direction and a width of Y in the vertical direction. The surrounding blocks correspond to an analysis window for calculating characteristic parameters. The displacements from the surrounding blocks to the reference block are d _x and d _y . The upper right of FIG. 5 shows the distribution of signal values in the reference patch oi52 in the reference block. The characteristic analysis unit 124 executes processing similar to steps S12 to S16 described above in order to generate reference patches for each block.

However, in step S12, the characteristic analysis unit 124 extracts (extracts) a portion defined from the image data using the surrounding blocks as an analysis window as the surrounding patch ri54.
In step S14, the characteristic analysis unit 124 calculates the frequencies f _xk ′, f _yk ′, and amplitude A for each frequency component k as characteristic parameters using equation (3), for example, for the surrounding patch oi54 extracted in step S12. _k ' and phase φ _k ' are calculated (analysis). The lower left of FIG. 5 shows the distribution of signal values in the surrounding patch ri54 before phase correction, which is represented by the calculated characteristic parameters.

In step S16, the characteristic analysis unit 124 adjusts the phase φ _k ′ of the characteristic parameters calculated for each frequency component k (phase correction), and adjusts the adjusted phase, frequencies f _xk ′, f _yk ′, and amplitude A′. A block-by-block estimated patch of a reference block including a frequency component k having a frequency component k is generated. When adjusting the phase φ _k ′ related to _the frequency component k, _the characteristic analysis unit 124 calculates _the wave numbers _f The inner product of the displacements d _x and d _y on the plane is added as an adjustment value to the phase φ _k ′ before correction. Then, the characteristic analysis unit 124 estimates the reference patch in the reference block using the sum of the frequency components k having the adjusted phase, frequencies f _xk ′, f _yk ′, and amplitude A _k ′ as shown in Equation (4). It can be generated as a value, that is, an estimated patch for each block.

In Equation (4), R(n _x , _ny ) indicates the signal value of the pixel located at the coordinates n _x , _ny of the estimated patch for each block. K is an integer of 2 or more indicating the number of frequency components. f _xi ′/f _sx and f _yi ′/f _sy correspond to wave numbers k _xi and k _yi of frequency component k, respectively. φ′+f _xk ′ d _x /f _sx +f _yk ′ _dy /f _sy corresponds to the adjusted phase of frequency component k. A signal including the calculated signal value R(n _x , _ny ) for each pixel is reconstructed as the estimated patch ri56.

FIG. 6 is a diagram showing another example of reconstructing the estimated signal. In the example shown in FIG. 6, characteristic parameters are calculated in the analysis block set at the center of the original image oi62, and estimated signals are reconstructed for each of eight adjacent blocks adjacent to the analysis block from the calculated characteristic parameters. Here is an example. In FIG. 6, the distribution of signal values is shown in shading. The brighter the area, the larger the signal value, and the darker the area, the smaller the signal value. The extended image ei64 at the upper right of FIG. 6 and the extended image ei66 at the lower right of FIG. 6 are examples in which characteristic parameters in the analysis block are calculated using FFT and NHA, respectively. The reconstructed images in the respective adjacent blocks of the extended image ei64 and the extended image ei66 are reconstructed using the phase adjusted based on the displacement from the analysis block to the adjacent block as described above. However, the distribution of signal values in the analysis block of the original image oi62 is shown in the portions corresponding to the analysis blocks of the extended image ei64 and the extended image ei66.

In the expanded image ei64 based on FFT, the pattern is noticeably discontinuous between blocks. This phenomenon indicates significant discontinuities in signal values across block boundaries. On the other hand, in the extended image ei66 based on NHA, the pattern does not become discontinuous between blocks, and continuity is maintained. This shows that even across block boundaries, spatially continuous signal values can be estimated without significant discontinuity in signal values. In other words, according to NHA, it is possible to estimate a signal value in a block of interest from signal values in other blocks having different parts with higher accuracy than FFT. Therefore, it is expected that the characteristic parameters calculated using NHA can reduce the influence of surrounding blocks as an analysis window and improve the estimation accuracy of the signal to be analyzed in the reference block. This is based on the fact that NHA has high frequency resolution and can express the characteristics of the signal to be analyzed using fewer parameters.

In this way, the signal processing device 10 according to the present embodiment generates the section-by-section estimated signal in the reference section using the characteristic parameters of the surrounding section, which is a section different from the reference section. Therefore, when the periodicity of the signal values forming the signal to be analyzed is high, according to NHA, it is possible to generate an estimated signal for each section with extremely high accuracy. In such a case, as illustrated in FIG. 7, image data showing a periodic pattern across the reference block 72 and its surrounding blocks on a two-dimensional plane is used as the signal to be analyzed. .

However, even if the periodicity of the signal value is lower, if the signal to be analyzed in the reference interval includes a frequency component that has a common frequency with that in the surrounding interval, at least for that frequency component, the NHA By using , it is possible to estimate with high accuracy. For example, the image representing the state of the skin surface illustrated in FIG. 8 represents a spatially irregular pattern, but the reference patch serving as the analysis target signal in the reference block 82 is the surrounding patch related to the surrounding surrounding blocks. Contains common frequency components between. Since the common frequency component is reproduced regardless of the high or low frequency, even high-frequency components with relatively high frequencies, such as the fine structure of skin tissue, are not lost as much as when using the discrete Fourier transform. Therefore, reproducibility is improved compared to conventional methods. In addition, according to the emphasis processing unit 128, by averaging the estimated patches for each block generated for each of a plurality of different surrounding blocks, the noise component is reduced relative to the signal component, and the noise component is reduced relative to the signal component. It is possible to compensate for missing frequency components in the estimated patch. Therefore, according to the present embodiment, even when the signal value of the signal to be analyzed fluctuates irregularly within a section, the estimation accuracy of the signal to be analyzed can be improved.

(emphasis filtering)
Next, an example of emphasis filtering processing according to this embodiment will be described. The emphasis processing unit 128 averages the signal values for each pixel forming the estimated patch for each block generated for each of the surrounding blocks corresponding to the plurality of surrounding sections, and calculates the average value obtained by averaging the signal values among the plurality of surrounding blocks as a signal value. An estimated patch having the following values is generated as an estimated signal. The averaging process can be viewed as a form of emphasis filter filtering. This is because the averaging process allows signal components included in signal values to be relatively emphasized and random noise components to be reduced.

The emphasis processing unit 128 may employ, for example, any method such as a simple average, a geometric average, or a weighted average as the averaging process. In the averaging process, the emphasis processing unit 128 may prioritize estimated patches for each block that are more similar to the reference patch. For example, the emphasis processing unit 128 selects as the averaging target a highly estimated patch for each block whose similarity with the reference patch is equal to or higher than a predetermined similarity, and rejects other estimated patches for each block. Limit the estimated patches for each block. In the weighted average of a plurality of block-based estimated patches, the emphasis processing unit 128 assigns a higher weight to the block-based estimated patches that are more similar to the reference patch, instead of limiting the block-based estimated patches, or in addition to limiting the block-based estimated patches. The weighting coefficient used for calculation (multiplication) during averaging may be increased.

When image data is used as the signal to be analyzed as described above, the emphasis processing unit 128 can apply, for example, the method described in Non-Patent Document 1 (hereinafter referred to as BM3D) in the averaging process. BM3D has the following steps S22-S30.
(Step S22) Grouping, (Step S24) 3D transform, (Step S26) Hard-thresholding, (Step S28) Inverse 3D transform, (Step S30) Aggregation

However, the emphasis processing unit 128 according to the present embodiment may omit step S22 and may employ the plurality of estimated patches for each block generated by the signal estimation unit 126. This is because step S22 is a step of searching for a similar patch similar to the reference patch within one frame of the image. The emphasis processing unit 128 may determine the similarity between the adopted estimated patch for each block and the reference patch, and select the estimated patch for each block to be adopted as a processing target from step S24 onwards. The emphasis processing unit 128 can use, for example, a least squared error (MSE) as a cost function that is an index value of the similarity, that is, the magnitude of the difference between the estimated patch for each block and the reference patch. . The emphasis processing unit 128 calculates the MSE for each estimated patch for each block, and ranks them in ascending order of MSE. Then, the emphasis processing unit 128 may select a block-wise estimated patch whose MSE is less than or equal to a predetermined upper limit value of MSE, or select a predetermined number of block-wise estimated patches in ascending order from the block-wise estimated patch whose MSE is the smallest. You may also select patches. The MSE between the two-dimensional estimated patch for each block and the reference patch can be calculated using Equation (5), for example. In the example shown in equation (5), the difference between the signal value s(n _{x , ny ) of the reference patch for each pixel n x , ny and} the signal value s'(n _x , _ny ) of the estimated patch for each block is It is obtained by dividing the sum of square values between pixels in a block by the number of pixels N _x N _y . MSE is a cost function corresponding to the L2 norm, which is an index value of the size of a vector or matrix. Note that step S30 is a process equivalent to the above-mentioned weighted average.

The processing of each step will be explained below.
(Step S24) The emphasis processing unit 128 generates one piece of three-dimensional image data by stacking the acquired estimated patches for each block, for example, according to a predetermined order. The emphasis processing unit 128 performs discrete Fourier transform on the generated three-dimensional image data and calculates frequency domain coefficients in the three-dimensional frequency domain.

(Step S26) The emphasis processing unit 128 performs forced threshold processing on each frequency domain coefficient. Forced threshold processing means that if the absolute value of the value to be processed is less than or equal to the upper limit of a predetermined absolute value, the value to be processed is adopted as is, and the absolute value of the value to be processed is less than the upper limit. If it exceeds the upper limit, the value to be processed (complex number) is attenuated so that its absolute value becomes the upper limit. Forced thresholding is also called clipping. This is because when the level of noise components mixed into the signal to be processed becomes significant, the absolute values of some frequency domain coefficients increase abnormally, so that they are emphasized more than the signal components in subsequent steps. This is to reduce the risk of getting lost. The upper limit of the absolute value may be set in advance to a value that is sufficiently larger than the known noise level.

(Step S28) The emphasis processing unit 128 performs inverse Fourier transform on the frequency domain coefficients after forced threshold processing to generate three-dimensional frequency domain data after forced threshold processing. Each layer of the three-dimensional image data after forced threshold processing includes a processed estimated patch for each block.
(Step S30) The emphasis processing unit 128 sets the inter-layer sum of multiplication values obtained by multiplying the signal value of a certain layer by the weighting coefficient of that layer for each pixel of the processed estimated patch for each block as a new signal value. and generate an estimated patch including the calculated signal value. For example, the emphasis processing unit 128 uses a value that is inversely proportional to the product of noise and power as a weighting coefficient related to the estimated patch for each block in each layer. However, the noise component included in the signal to be processed may generally be unknown. Therefore, instead of the power of the noise component, the emphasis processing unit 128 may use the MSE of the estimated patch for each block of the layer and the reference patch. Assuming that the signal component and noise component in the generation of the estimated patch are uncorrelated, the more noise components there are, the higher the ratio of the noise component to MSE will be.If the estimation accuracy is sufficiently high, that is, the estimated patch for each block This is because if the reference patch sufficiently approximates the reference patch, the MSE is approximately proportional to the power of the noise component.

Note that the emphasis processing unit 128 may apply Wiener filtering, which is an example of emphasis filtering processing, to the estimated patches for each block. Wiener filtering assumes that signal components and noise components are uncorrelated, and uses a filter to minimize the power of the difference between the original signal to be analyzed and the filtered interval-specific estimated signal obtained by multiplying the filter coefficients in the frequency domain. This is a method of determining coefficients. For example, in Wiener filtering, the emphasis processing unit 128 multiplies a frequency domain transform coefficient by a filter coefficient. The emphasis processing unit 128 generates filtered three-dimensional image data by performing inverse discrete Fourier transform on the filtered three-dimensional frequency domain data, which has a multiplication value obtained by multiplying the filter coefficients as a filtered transform coefficient. The emphasis processing unit 128 determines filter coefficients so that the power of the difference between the reconstructed patch extracted from the filtered three-dimensional image data and the reference patch is minimized. According to Wiener filtering, the filter coefficients are determined so that the contribution of the surrounding patches based on surrounding blocks having a larger difference from the signal to be analyzed in the reference block is smaller, so that noise components are reduced. Furthermore, since frequency components missing or suppressed from the reference patch are compensated for in each estimated patch for each block, the estimation accuracy of the reference patch can be further improved.

(Noise suppression processing)
An example of noise suppression processing according to this embodiment will be described below. FIG. 9 is a flowchart illustrating an example of noise suppression processing according to this embodiment. In the example shown in FIG. 9, image data representing one frame of a plane image is used as a signal to be analyzed, and NHA is used to calculate characteristic parameters.

(Step S102) The characteristic analysis unit 124 sets a reference block that is a region of interest as a processing target in one frame. Further, the characteristic analysis unit 124 sets a plurality of surrounding blocks, which are areas having the same size as the reference block but different from the reference block. The surrounding block may be an area that does not overlap with the reference block, or may be an area that partially overlaps with the reference block.

(Step S104) The characteristic analysis unit 124 determines whether there is an unprocessed surrounding block in the image data to be analyzed, and when determining that there is an unprocessed surrounding block (step S104 YES), the characteristic analysis unit 124 performs the processing of step S106. Proceed to. When it is determined that there are no unprocessed surrounding blocks (step S104 NO), the process proceeds to step S118.

(Step S106) The characteristic analysis unit 124 identifies any one of the unprocessed surrounding blocks as a surrounding block to be processed, and extracts a surrounding patch indicating a partial image within the identified surrounding block from the image data. Thereafter, the characteristic analysis unit 124 sets the initial value of the number of analyzed frequency components to 0, and proceeds to the process of step S108.

(Step S108) The characteristic analysis unit 124 determines whether the number of analyzed frequency components is less than K. When it is determined that the number is less than K (step S108 YES), the process advances to step S110. When it is determined that the number is K or more (NO in step S108), the process advances to step S114.

(Step S110) The characteristic analysis unit 124 performs NHA on the surrounding patches including signal values in the surrounding blocks, and uses the characteristic parameter of the frequency component whose residual difference with the processing target signal is minimized as a characteristic parameter. , define a two-dimensional set of frequency, amplitude and phase. However, the surrounding patch is used as the initial value of the signal to be processed. Thereafter, the process advances to step S112.

(Step S112) The characteristic analysis unit 124 subtracts, for each pixel, the estimated signal value by frequency of the frequency component configured based on the characteristic parameter determined in step S110 from the signal value of the signal to be analyzed. A new analysis target signal having the frequency component residual as a signal value is generated. Then, the characteristic analysis unit 124 adds 1 to the number of analyzed frequency components. Thereafter, the process advances to step S108.

(Step S114) The signal estimation unit 126 corrects the calculated phase based on the frequency calculated for each frequency component k and the displacement from the surrounding blocks to the reference block (phase correction). The signal estimation unit 126 generates a block-specific estimated patch obtained by combining (adding) frequency-specific estimated signals for each frequency component k in surrounding blocks obtained using a set of frequencies, amplitudes, and corrected phases ( presume. Thereafter, the process advances to step S116.

(Step S116) The characteristic analysis unit 124 excludes the surrounding blocks to be analyzed from the unprocessed surrounding blocks. Thereafter, the process advances to step S104.
(Step S118) The emphasis processing unit 128 sorts the order of the individual estimated patches based on the magnitude of the difference between the block-specific estimated patch for each surrounding block generated by the signal estimation unit 126 and the reference patch. For example, the emphasis processing unit 128 calculates MSE as a size index value, and determines the order of estimated patches for each block in ascending order of MSE.
(Step S120) The emphasis processing unit 128 performs emphasis filtering on each estimated patch for each block to generate an estimated patch. The emphasis processing unit 128 performs the processes of steps S24 to S30 described above in emphasis filtering. Thereafter, the process shown in FIG. 9 ends.

Note that the characteristic analysis unit 124 may set the positions of the individual surrounding blocks so that they are arranged without overlapping each other in the entire area other than the reference block in the image of one frame in step S102; It is not limited to this. The characteristic analysis unit 124 may set the positions of the individual surrounding blocks so that they are arranged without overlapping each other within a predetermined range from the reference block, which is a part of the image of one frame. For example, when the size of the reference block and the surrounding blocks are 8 pixels in the horizontal direction and 8 pixels in the vertical direction (hereinafter referred to as 8×8), the characteristic analysis unit 124 analyzes 40 pixels having the same center of gravity as the reference block. The area may be set within the range of ×40.

(Example of reconstructed image)
Next, an example of a reconstructed image constructed by arranging estimated patches generated for each of a plurality of reference blocks in a plane image of one frame in a two-dimensional plane by the process illustrated in FIG. I will explain about it. However, the sizes of the reference block and surrounding blocks were each 8×8. The sizes of the reference block and surrounding blocks can be set arbitrarily, but by making them small, the signal components of each block become mainly low-frequency components with relatively low frequencies. Therefore, deterioration in estimation accuracy due to irregular patterns can be suppressed. Further, the number K of frequency components was set to 30. The number K of frequency components can also be set arbitrarily, but the estimation accuracy tends to increase as the number K of frequency components increases. For example, if the size of the reference block is 8×8, and the number of frequency components K is 5-20, the PSNR of the estimated patch (described later) can be calculated from the characteristic parameters obtained by performing NHA on the surrounding patches. It becomes 40dB or more. This estimation accuracy corresponds to sufficiently good quality that the patch cannot be distinguished from the reference patch by human vision.

FIG. 10 shows a reconstructed image p1004 generated using BM3D as a conventional method, and a reconstructed image p1004 generated using BM3D as a conventional method, using a reference image that is a part of a one-frame planar image showing the state of a human skin surface as an original image o1002. The reconstructed image p1006 indicated by the estimated signal generated by the process shown in FIG. 9 is shown. The reconstructed image p1004 shows the same overall change in brightness as the original image o1002, but the brightness has been smoothed and minute changes in brightness have become unclear. This is because high frequency components with relatively high frequencies are lost during the processing process. On the other hand, in the reconstructed image p1006, almost no brightness smoothing occurs, and the fine changes in brightness that appear in the original image o1002 are maintained. This shows that according to this embodiment, high frequency components are not lost and the original image can be reconstructed with higher accuracy than the conventional method.

Next, an example of a reconstructed image generated using a noise-added image obtained by adding noise to a standard image as an analysis target signal will be described. Here, the standard image means an image having standard resolution. The size of the reconstructed image was set to 512×512. However, in the examples shown in FIGS. 11 and 12, a green pepper exhibiting an overall smooth pattern was used as the subject. In addition, the surface of a boat, where minute changes in shading appear regularly, was used as the subject.

The original image o1202 shown in FIG. 12 corresponds to the image within the attention area o1102 set approximately to the right of the center in FIG. In FIG. 12, reconstructed images p1206 and p1208 are images obtained by integrating estimated patches generated using BM3D and the method of this embodiment, respectively, for the noise-added image n1204. Also in the example shown in FIG. 12, the pattern of the original image o1202 is reproduced more clearly in the reconstructed image p1208 than in the reconstructed image p1206. In particular, in the reconstructed image p1208, the same pattern as the original image o1202 is reproduced without blurring the unevenness of the stem or the reflection of the irradiated light. Therefore, the estimation accuracy of the reconstructed image was quantitatively evaluated using two methods, σ5 and σ10, as the level of noise added to the original image, and two methods, BM3D and this embodiment (four methods in total). Here, σ5 means Gaussian noise that is a series of random signal values with a standard deviation of 5. Values such as 5 and 10 are dimensionless values that serve as a guide for the amplitude of a signal value, and are not absolute values.

FIGS. 13 and 14 are tables showing examples of estimation accuracy of reconstructed images of green peppers and boats, respectively. As estimation accuracy, PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) were determined. PSNR and SSIM are index values indicating that the larger the PSNR and SSIM are, the higher the reproducibility and visual similarity of the reconstructed image. According to the examples shown in FIGS. 13 and 14, for both noise levels σ5 and σ10, the PSNR and SSIM of the reconstructed image generated using this embodiment are larger than those of BM3D. This shows that according to this embodiment, the estimation accuracy of the reconstructed image can be quantitatively improved. Furthermore, the lower the noise level is, the more remarkable the PSNR and SSIM according to this embodiment tend to increase.

Next, an example of a reconstructed image generated using a noise-added image obtained by adding noise to a high-resolution image as an analysis target signal will be described. Here, a high-resolution image means an image having a significantly higher resolution than a standard resolution. 15 and 16 are examples of 2K images in which the number of pixels in the horizontal direction of the entire image is 2048. However, the size of the reconstructed image was set to 512×512. In the example shown in FIG. 15, the surface of a boat was used as the subject. In the example shown in FIG. 15, the front of a classic car with letters and prominent outlines (edges) of straight lines and arcs was used as the subject.

In FIG. 15, reconstructed images p1506 and p1508 are reconstructed images obtained by reconstructing a noise-added image n1504 obtained by adding noise to an original image o1502 using BM3D and the method of this embodiment, respectively. It is. In FIG. 16, reconstructed images p1606 and p1608 are reconstructed images obtained by reconstructing a noise-added image n1604 obtained by adding noise to an original image o1602 using BM3D and the method of this embodiment, respectively. It is.

In the example shown in FIG. 15, the reconstructed image p1508 has a better pattern appearing in the original image than the reconstructed image p1506, and in the example shown in FIG. vividly reproduced. In the reconstructed image p1508, a fine pattern is reproduced without blurring the unevenness of the boat surface. In the reconstructed image p1608, a clear pattern similar to the original image is reproduced without blurring around the characters or edges. We also quantitatively evaluated the estimation accuracy of reconstructed images for high-resolution images. Therefore, regarding σ5 as the level of noise added to each of the original images o1502 and o1602 shown in FIGS. 15 and 16, we quantified the estimation accuracy of the reconstructed image using two methods (BM3D and this embodiment) (four methods in total). evaluated.

FIG. 17 is a table showing an example of estimation accuracy of reconstructed images. According to the example shown in FIG. 17, for both the original images o1502 and o1602, the PSNR and SSIM of the reconstructed image generated using this embodiment are larger than those of BM3D. Furthermore, the degree of improvement in the estimation accuracy of the reconstructed image according to this embodiment is more remarkable for the original image o1602 than for the original image o1502. This confirms that the improvement in image quality in the original image o1602, which includes many edges having high-frequency components with high spatial frequencies, makes a large contribution.

As described above, the signal processing device 10 according to the present embodiment calculates characteristic parameters indicating the characteristics of the analysis target signal from the analysis target signal in each of the plurality of first intervals (for example, reference intervals). An analysis section (eg, characteristic analysis section 124) is provided. The signal processing device 10 includes an estimator (for example, a signal estimator) that generates an estimated signal for each section including an estimated value of the signal to be analyzed in a second section (for example, a reference section) based on characteristic parameters for each of the first sections. 126), and a processing unit (for example, an emphasis processing unit 128) that generates an estimated signal by averaging the estimated signals for each section of the first section. According to this configuration, it is possible to average the section-by-section estimated signals estimated for each of the first sections to generate an estimated signal that includes the estimated value of the analysis target signal of the second section. Noise components included in the section-wise estimated signals of the individual first sections are relatively reduced by averaging. Furthermore, an estimated signal is generated in which suppressed or missing components are complemented for the section-specific estimated signal of each first section. Therefore, noise can be suppressed while maintaining each component constituting the signal to be analyzed.

In addition, the analysis section calculates the amplitude and phase of each frequency component as characteristic parameters, and the estimation section calculates the phase correction value (i.e., the correction value after correction) based on the displacement from each of the first sections to the second section for each frequency component. ), calculate the reconstructed value in the second section based on the amplitude and phase correction values, and synthesize the frequency-specific estimated signals including the calculated reconstructed values to generate the section-specific estimated signal. Good too. According to this configuration, the frequency components of the signal to be analyzed in the second section, which is different from the first section, are reconstructed using the phase corrected based on the displacement from the first section to the second section for each frequency component. be able to.

In addition, the analysis unit calculates reconstructed values in each of the first sections based on the frequency, amplitude, and phase for each frequency component, and combines the frequency-specific estimated signal including the reconstructed values with the analysis target signal in each of the first sections. The frequency, amplitude, and phase of each frequency component may be calculated so that the difference between the two frequencies is minimized. According to this configuration, it is possible to generate the section-wise estimated signal obtained by synthesizing the frequency-wise estimated signals of the second section with high precision using a small number of characteristic parameters. Therefore, the signal to be analyzed in the second section can be estimated with high accuracy.

Further, the processing unit may select at least two section-specific estimated signals to be averaged, giving priority to the smaller the difference from the analysis target signal in the second section among the section-specific estimated signals. According to this configuration, since it is possible to preferentially use the section-specific estimated signal with high reproducibility of the analysis target signal for averaging, it is possible to improve the estimation accuracy of the analysis target signal.

In addition, the processing unit calculates the estimated signal by weighted averaging the estimated signals for each section in the first section, and the larger the difference from the analysis target signal in the second section, the more the estimated signal for each section in the first section increases. Coefficients for each section of the estimated signal in the first section in the weighted average may be determined so that the contribution of the signal to the estimated signal is small. According to this configuration, the estimation accuracy of the analysis target signal can be improved because the weighted average gives more weight to the section-specific estimated signal with higher reproducibility of the analysis target signal.

Furthermore, the signal to be analyzed is an image signal indicating a signal value for each pixel in a two-dimensional image, and each of the plurality of first sections and second sections is a partial region (for example, a block) of a different two-dimensional image. It may be. According to this configuration, the signal processing device and the signal processing method according to the present embodiment can suppress noise while improving the estimation accuracy of the second section, which is a partial region of a two-dimensional image.

Although one embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to that described above, and various design changes etc. may be made without departing from the gist of the present invention. It is possible to

For example, in the embodiments described above, image data representing a planar image, which is a type of two-dimensional signal, is used as the analysis target signal, but the present invention is not limited to this. The signal to be analyzed may be a one-dimensional signal or data having three or more dimensions. In addition, the signals to be analyzed are not limited to those having physical quantities such as audio signals, acceleration signals, biological signals, temperature, and atmospheric pressure as signal values, but also the prices of securities such as stocks and various goods, exchange rates, etc. , it may be a series having an index value indicating a social phenomenon such as the population of a certain area as a signal value. In addition, the characteristic parameters are not limited to NHA, but also relate to mathematical models that can express the characteristics of the signal to be analyzed and can estimate the signal to be analyzed with high accuracy in a surrounding interval separate from the reference interval based on the calculated characteristic parameters. It may be a parameter. For example, for a signal to be analyzed in which the characteristics of a signal value can be regressed with high precision using a linear function as a mathematical model, the intercept and slope correspond to the characteristic parameters.

The signal to be analyzed is not limited to a series of signal values sampled at equal intervals in time or space, but may also be a series of signal values sampled at unequal intervals.
The averaging processing method is not limited to a method that applies BM3D, as long as the signal component can be emphasized relatively more than the noise component, and other methods such as non-local averaging filtering method (NLM) can be used. A method applying local means filtering may also be used. In NLM as well, the above estimated patch for each block may be applied instead of the patch surrounding a part of the image. Furthermore, the targets of the averaging process are not limited to the section-by-section estimated signals of individual surrounding sections, but may also include analysis target signals of reference sections. Even in that case, the noise component that is synthesized by averaging the noise component included in the signal to be analyzed in the reference section and the noise component included in the estimated signal for each section is relatively reduced compared to the signal component that is synthesized. can do.

When image data representing a two-dimensional image or a three-dimensional image is used as the signal to be analyzed, the present embodiment may be applied as an elemental technology in computer graphics. For example, in addition to removing noise components in editing videos such as movies, game content, and broadcast program videos, it may also be used to interpolate and extend (extrapolation processing) missing portions. In that case, the control unit 120 may specify the area to be interpolated or expanded as a reference block.

DESCRIPTION OF SYMBOLS 10... Signal processing device, 102... Processor, 104... ROM, 106... RAM, 108... Operation input section, 110... Display section, 112... Input/output section, 120... Control section, 122... Signal acquisition section, 124... Characteristic analysis unit, 126...signal estimation unit, 128...emphasis processing unit, 140...storage unit

Claims (7)

an analysis unit that calculates a characteristic parameter indicating a characteristic of the analysis target signal from the analysis target signal in each of the plurality of first sections;
an estimation unit that generates an estimated signal for each section including an estimated value of the signal to be analyzed in a second section based on the characteristic parameter for each of the first sections;
a processing unit that averages the estimated signals for each section of the first section to generate an estimated signal ;
The analysis unit calculates the amplitude and phase of each frequency component as the characteristic parameters,
The estimation unit calculates the phase correction value based on the displacement from each of the first sections to the second section for each frequency component, and calculates the phase correction value based on the amplitude and the phase correction value. Calculate the reconstructed value in the interval,
The frequency-based estimated signals including the reconstructed values are combined to generate the section-based estimated signal.
Signal processing device. The analysis unit calculates a reconstruction value in each of the first sections based on the frequency, amplitude, and phase of each frequency component,
The frequency, amplitude, and phase of each frequency component are calculated so that the difference between the frequency-specific estimated signal including the reconstructed value and the analysis target signal in each of the first sections is minimized. signal processing device. The processing unit selects at least two section-specific estimated signals to be averaged, giving priority to smaller differences from the analysis target signal in the second section from among the section-specific estimated signals. The signal processing device according to claim 1 or claim 2 . The processing unit calculates the estimated signal by weighted averaging the estimated signals for each section of the first section,
Each of the first intervals in the weighted average is set such that the larger the difference between the second interval and the analysis target signal, the smaller the contribution of each interval estimated signal of the first interval to the estimated signal. The signal processing device according to any one of claims 1 to 3 , wherein a coefficient for an estimated signal for each section is determined. The analysis target signal is an image signal indicating a signal value for each pixel in a two-dimensional image, and the plurality of first sections and the plurality of second sections are respective different partial regions of the two-dimensional image. The signal processing device according to any one of claims 1 to 4 . A signal processing method in a signal processing device, the method comprising:
a first step of calculating a characteristic parameter indicating a characteristic of the signal to be analyzed from the signal to be analyzed in each of the plurality of first sections;
a second step of generating an estimated signal for each section including an estimated value of the signal to be analyzed in a second section based on the characteristic parameter for each of the first sections;
a third step of generating an estimated signal by averaging the estimated signals for each section of the first section ,
In the first step, the amplitude and phase of each frequency component are calculated as the characteristic parameters,
The second step includes calculating the phase correction value based on the displacement from each of the first sections to the second section for each frequency component, and calculating the phase correction value based on the amplitude and phase correction values. Calculate the reconstructed values in two sections,
The frequency-based estimated signals including the reconstructed values are combined to generate the section-based estimated signal.
Signal processing method. In the computer of the signal processing device,
a first step of calculating a characteristic parameter indicating a characteristic of the analysis target signal from the analysis target signal in each of the plurality of first sections;
a second step of generating an estimated signal for each section including an estimated value of the signal to be analyzed in a second section based on the characteristic parameter for each of the first sections;
a third step of generating an estimated signal by averaging the estimated signals for each section of the first section;
A program for executing
The first step is to calculate the amplitude and phase of each frequency component as the characteristic parameters,
The second step is to calculate the phase correction value based on the displacement from each of the first sections to the second section for each frequency component, and calculate the phase correction value based on the amplitude and phase correction values. Calculate the reconstructed values in two sections,
The frequency-based estimated signals including the reconstructed values are combined to generate the section-based estimated signal.
program.

Images