JP2001136547A - Signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a signal processor which can obtain an equivalent image as usual in a short processing time by reducing the operation quantity of high- range information. SOLUTION: Among signal components from photoelectric conversion unit areas having a 1st spectral sensitivity characteristics among spectral sensitivity characteristics and signal components from photoelectric conversion unit areas having other 2nd and succeeding spectral sensitivity characteristics, respective DCT coefficients are calculated (A14), and a correlation coefficient is calculated between a DC component of the obtained DCT coefficient of the photoelectric conversion areas having the 1st spectral sensitivity characteristics and a DC component of the DCT coefficient of the photoelectric conversion areas having the 2nd and succeeding spectral sensitivity characteristics which are small in information amount. According to the obtained correlation coefficient and an AC component of the DCT coefficient of the photoelectric conversion areas, having the 1st spectral sensitivity characteristics, an AC component of the DCT coefficient of the photoelectric conversion areas having the 2nd and succeeding spectral sensitivity characteristics, is generated (A16).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing device.

[0002]

2. Description of the Related Art As an example of a signal processing device, there is a decompression device for decompressing a compressed image. In the image data output in the process of expanding the compressed image using such an expansion device, a red (hereinafter referred to as R) or blue (hereinafter referred to as B) color signal having a low spatial sampling frequency includes a spatial signal. There is a green (hereinafter referred to as G) luminance signal which is a signal having a high sampling frequency and closest to the luminance signal among the three colors. Therefore, in many cases, it is possible to estimate the high-frequency information of the R and B color signal components based on the high-frequency information of the G luminance signal after frequency-decomposing the input signal by, for example, wavelet transform. By performing the inverse wavelet transform after performing such high-frequency information estimation (high-frequency estimation), a color signal in which high-frequency information has been recovered can be obtained.

FIG. 12 is a diagram for explaining a conventional signal processing procedure in which high-frequency estimation and image compression processing are individually performed. The signal processing method shown in FIG.
-284798.

An original image having a different information amount depending on the color (A1) input by the image input means of the signal processing device is subjected to pixel interpolation processing (A2), and then subjected to an R signal frame memory,
It is stored in the frame memory for G signal and the frame memory for B signal respectively (A3). Next, each of the R, G, and B signals is decomposed into a low frequency component and a high frequency component by a signal conversion method such as a wavelet transform (A4). Next, a correlation coefficient is calculated between the low frequency component of the R signal and the low frequency component of the G signal (A
5). Next, the high frequency component of the R signal is estimated based on the calculated correlation coefficient and the high frequency component of the G signal (A6).

Next, the R signal estimated as the low frequency component of the R signal
A high-definition R signal is obtained by synthesizing the high frequency component of the signal by inverse wavelet transform or the like (A7). The above is R
Although the signal has been described, a high-definition signal can be obtained for the B signal by the same method.

Next, DCT is applied to the R signal obtained by the synthesis.
(Discrete Cosine Transformation) processing and DC of R or Cr
Obtain the T coefficient. Similarly, DCT processing is performed on the B signal to obtain a DCT coefficient of B or Cb. Further, the G signal is D
A CT process is performed to obtain a G or Y DCT coefficient (A
8). Next, a compression process is performed using the obtained DCT coefficients (A9), and the data is recorded on a recording medium or transmitted to the outside (A10).

A procedure for estimating the high frequency when the compressed image data recorded or transmitted by the above method is expanded will be described with reference to FIG. The compressed image data recorded on the recording medium or the compressed image data transmitted to the outside is read from the recording medium or received (B1) on the receiving side, and then decompressed using the DCT coefficient to obtain Y, Cr, Cb
(Or R, G, B) signals (B2). next,
Each of the R, G, and B signals is decomposed into a low frequency component and a high frequency component by a signal conversion method such as a wavelet transform (B3). Next, a correlation coefficient is calculated between the low frequency component of the R signal and the low frequency component of the G signal (B4). Next, the calculated correlation coefficient and G
The high frequency component of the R signal is estimated based on the high frequency component of the signal (B5).

Next, the R signal estimated as the low frequency component of the R signal
A high-definition R signal is obtained by synthesizing the high frequency component of the signal by inverse wavelet transform or the like (B6). The above is R
Although the signal has been described, a high-definition signal can be obtained for the B signal by the same method. Next, the R and B signals obtained by frequency synthesis and the G signal obtained by decompression processing are stored in respective frame memories (B
7), the images are synthesized and become a reproduced image (B8).

[0009]

However, the prior art including Japanese Patent Application Laid-Open No. 09-284798 has the following problems.

First, since the estimation of the high-frequency component and the compression or decompression processing using the DCT coefficient are performed separately, the processing of the high-frequency information is separately performed, thereby increasing the processing time. There is a problem.

Further, when high-frequency components are generated between pieces of predetermined area information having different numbers of pixels, a false color is generated in a rapidly changing portion of data due to a difference in the position of the center of gravity of a local area.

In addition, when the correlation coefficient and the similarity are calculated for the entire image, if the range in which the high-frequency component exists is small in the entire image, the high-frequency component is not generated.

If the subject has less high-frequency information of G than high-frequency information of R or B, if pixel interpolation processing using high-frequency information generation is performed, the high-frequency information decreases and the image is reduced. The definition of is worse.

If the high-frequency information is accurately obtained, the amount of calculation is large and the actual processing time is long.

Further, even when the variance of the amplitude is used, the amount of calculation is large and the actual processing time is long.

When there is little high-frequency information in the subject, there is no large difference in the output result between interpolation based on estimation of high-frequency information and interpolation without estimation of high-frequency information. , The amount of calculation for the interpolation is large, and the actual processing time becomes long.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problem, and an object of the present invention is to obtain an image equivalent to a conventional image in a short processing time by reducing the amount of calculation of high-frequency information. It is an object of the present invention to provide a signal processing device capable of performing the above.

It is another object of the present invention to provide a signal processing apparatus capable of suppressing the occurrence of a false color in a rapidly changing portion of data due to a difference in the position of the center of gravity of a local area.

Another object of the present invention is to provide a signal processing apparatus in which the generation of high-frequency components is not performed even though high-frequency components need to be estimated.

Another object of the present invention is to reduce the high-frequency information even when the high-frequency information of G is smaller than the high-frequency information of R or B. It is an object of the present invention to provide a signal processing device that can obtain an equivalent image with a small amount of computation when interpolation such as cubic or linear is performed when the definition of the image does not deteriorate and the information amount is the same.

Another object of the present invention is to make almost the same judgment as compared with a case where all the high frequency information is compared.
An object of the present invention is to provide a signal processing device which requires a small amount of calculation and can be realized in a short processing time.

Another object of the present invention is to provide a signal processing apparatus which can make almost the same judgment as compared with the case where the variance of amplitude is used, has a small amount of calculation, and can be realized in a short processing time. It is in.

It is another object of the present invention to provide a signal processing apparatus capable of realizing signal processing with a small amount of calculation when there is no large difference in output results.

[0024]

In order to achieve the above object, a signal processing apparatus according to a first aspect of the present invention has image input means having a plurality of spectral sensitivity characteristics, and has at least one signal relating to at least one spectral sensitivity characteristic. In a signal processing apparatus for processing a signal whose information amount is larger than the information amount of a signal related to another spectral sensitivity characteristic, the number of photoelectric conversion unit regions having the first spectral sensitivity characteristic among the plurality of spectral sensitivity characteristics But the second
Image input means which is a plurality of times the number of photoelectric conversion unit areas having other spectral sensitivity characteristics that are the second and subsequent, and signal components of the photoelectric conversion area having the first spectral sensitivity characteristic input by the image input means And, from the signal component of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics with a small amount of information,
DCT coefficient calculating means for calculating each DCT coefficient; DCT coefficients of DCT coefficients of the photoelectric conversion region having the first spectral sensitivity characteristic obtained by the DCT coefficient calculating means; Correlation coefficient calculating means for calculating a correlation coefficient between the DCT coefficient of the DCT coefficient of the photoelectric conversion region having the spectral sensitivity characteristic of the following, a correlation coefficient obtained from the correlation coefficient calculating means, Based on the AC component of the DCT coefficient of the photoelectric conversion region having the first spectral sensitivity characteristic obtained by the coefficient calculating means, the DCT coefficient of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information is obtained. And an AC component signal generating means for generating an AC component of the photoelectric conversion region having the first spectral sensitivity characteristic.
The DC and AC components of the T coefficient and the DC and AC components of the DCT coefficient of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information are compressed or expanded using the calculated DCT coefficients. Compression / decompression means for performing processing.

Further, the signal processing device according to the second invention is characterized in that:
In the first invention, there is provided interpolation means for equalizing the centers of gravity of pixels of each spectral sensitivity characteristic in a predetermined area of the two DC component signals input to the correlation coefficient calculation means.

Further, the signal processing apparatus according to the third aspect of the present invention
In the first aspect, the image processing apparatus further includes an image decomposing unit that decomposes one image into a plurality of blocks, and the AC component signal generating unit generates an AC component signal for each block decomposed by the image decomposing unit.

Further, a signal processing device according to a fourth aspect of the present invention comprises:
In the first invention, an AC component signal of a DCT coefficient of a photoelectric conversion region having a first spectral sensitivity characteristic and an AC component of a DCT coefficient of a photoelectric conversion region having a second or subsequent spectral sensitivity characteristic having a small amount of information. An AC component signal comparing means for comparing the signal with the signal; an interpolating means which does not depend on the estimation of the AC component signal such as cubic or linear; An AC component estimation method switching unit that adaptively switches between component signal estimation processing and interpolation processing such as cubic or linear is provided.

The signal processing device according to a fifth aspect of the present invention
In the fourth aspect, the AC component signal comparing means includes a variance calculating means for calculating a variance value of an AC component signal of a DCT coefficient from a photoelectric conversion region having first, second and subsequent spectral sensitivity characteristics, Comparing means for comparing the obtained variance values.

Further, the signal processing device according to the sixth invention comprises:
In the fourth invention, the AC component signal comparing means calculates a difference between the maximum value and the minimum value of the AC component signal of the DCT coefficient from the photoelectric conversion regions having the first and second or later spectral sensitivity characteristics. An arithmetic means and a comparing means for comparing the obtained difference between the maximum value and the minimum value are provided.

The signal processing device according to a seventh aspect of the present invention
In the fourth invention, the AC component signal comparing means includes a DCT from a photoelectric conversion region having a first spectral sensitivity characteristic.
Dispersion calculation means for calculating the variance value of the AC component signal of the coefficient, and variance value comparison means for comparing the obtained variance value with a threshold value are provided.

Further, the signal processing apparatus according to the eighth aspect of the present invention
In the fourth invention, the AC component signal comparing means includes a DCT from a photoelectric conversion region having a first spectral sensitivity characteristic.
An arithmetic unit for calculating the difference between the maximum value and the minimum value of the AC component signal of the coefficient, and a comparing unit for comparing the difference between the maximum value and the minimum value of the AC component signal with a threshold value.

[0032]

Embodiments of the present invention will be described below in detail with reference to the drawings. A signal processing device to which the present invention is applied has image input means having a plurality of spectral sensitivity characteristics, and a signal amount of a signal relating to at least one spectral sensitivity characteristic is larger than an information amount of a signal relating to another spectral sensitivity characteristic. And among the plurality of spectral sensitivity characteristics, the number of photoelectric conversion unit regions having the first spectral sensitivity characteristic (that is, the number of pixels of the photoelectric conversion region) is
An image input unit having a plurality of times the number of photoelectric conversion unit regions having the second and subsequent spectral sensitivity characteristics, and a photoelectric conversion region having the first spectral sensitivity characteristic input by the image input unit. DCT coefficient calculation means for calculating respective DCT coefficients from the signal components and the signal components in the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information, and the first DCT coefficient obtained by the DCT coefficient calculation means Of the DCT coefficient of the photoelectric conversion region having the second spectral sensitivity characteristic
Correlation coefficient calculating means for calculating a correlation coefficient between the component and the DC component of the DCT coefficient of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information, and the correlation coefficient calculating means On the basis of the correlation coefficient obtained from the above and the AC component of the DCT coefficient of the photoelectric conversion region having the first spectral sensitivity characteristic obtained by the DCT coefficient calculating means. AC component signal generating means for generating an AC component of a DCT coefficient of a photoelectric conversion region having a spectral sensitivity characteristic;
The DCT coefficient and the DC component of the CT coefficient and the DC and AC components of the DCT coefficient of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information are compressed or decompressed using the calculated DCT coefficient. Compression / decompression means for performing processing.

Here, a signal in which the information amount of the signal relating to at least one spectral sensitivity characteristic is larger than the other spectral sensitivity characteristics is, for example, a G signal, and another spectral response is based on the high-frequency information of the G luminance signal. It is possible to estimate high-frequency information of R and B color signal components as a signal relating to sensitivity characteristics.

FIG. 1 is a diagram for explaining a signal processing procedure according to the embodiment of the present invention. The present embodiment is characterized in that estimation of a high frequency component (AC component) and compression or expansion processing are performed simultaneously.

The original images having different amounts of information according to the colors (A11) input by the image input means of the signal processing device are subjected to pixel interpolation processing (A12), and then subjected to R signal frame memory, G signal frame memory, and B signal frame memory. Each is stored in the signal frame memory (A13). Next, DC is added to the R signal.
Perform T (Discrete Cosine Transform) processing to obtain R or Cr D
Obtain CT coefficients. Similarly, DCT processing is performed on the B signal to obtain a DCT coefficient of B or Cb. Further, a DCT process is performed on the G signal to obtain a G or Y DCT coefficient (A
14). Next, a correlation coefficient is calculated between the DC component of the DCT coefficient of the R or Cr signal and the DC component of the DCT coefficient of the G or Y signal, respectively (A15). Next, the AC component of the DCT coefficient of the R or Cr signal is estimated based on the calculated correlation coefficient and the AC component of the DCT coefficient of the G or Y signal (A16). The AC component of the DCT coefficient of the B or Cb signal can be estimated in a similar manner.

Next, the DCT coefficient D of the R or Cr signal is calculated.
C and AC components and DC of DCT coefficient of G or Y signal,
AC component, DC or AC of DCT coefficient of B or Cb signal
A compression process using a DCT coefficient is performed on the component (A1
7). The compressed image data obtained by the above method is recorded on a recording medium or transmitted to the outside (A18).

A procedure for performing high-frequency estimation when decompressing the compressed image data recorded or transmitted by the above method will be described below with reference to FIG. The compressed image data recorded on the recording medium or the compressed image data transmitted to the outside is read out from the recording medium or received on the receiving side (B1).
After 1), a pixel interpolation process (B12) is performed. Next, a correlation coefficient is calculated between the DC component of the DCT coefficient of the R or Cr signal and the DC component of the DCT coefficient of the G or Y signal (B13). Next, the AC component of the DCT coefficient of the R or Cr signal is estimated based on the calculated correlation coefficient and the AC component of the DCT coefficient of the G or Y signal (B14). The AC component of the DCT coefficient of the B or Cb signal can be estimated in a similar manner.

Next, the DCT coefficient D of the R or Cr signal is calculated.
C and AC components and DC of DCT coefficient of G or Y signal,
AC component, DC or AC of DCT coefficient of B or Cb signal
The extension processing is performed on the components (B15). The expanded color signals are stored in respective frame memories (B
16), and are combined into a reproduced image (B17).

FIG. 3 shows the conventional signal processing procedure (FIG. 3A) and the signal processing procedure according to the present embodiment (FIG. 3A) in which the estimation of the high-frequency component and the compression or decompression processing are performed simultaneously.
(B)) of FIG. As shown in the figure, in the past, high-frequency estimation and image compression or decompression processing were individually performed, but in the present embodiment, high-frequency component estimation and compression or decompression processing are performed simultaneously, so that high-frequency information The amount of calculation is reduced, and an image equivalent to the conventional image can be obtained in a short processing time.

In this embodiment, in estimating the high-frequency component, a processing block selected from an image on one screen as shown in FIG. FIG. 4B). FIG. 4C shows a configuration of DCT coefficients obtained by the DCT transform, and includes a DC component located in the upper leftmost block and AC components of other blocks.

The pixel interpolation processing described above (A1 in FIG. 1)
2. Details of B12) in FIG. 2 will be specifically described. Here, the pixel interpolation processing includes a center-of-gravity alignment processing, a block decomposition processing, and a switching processing that switches between an interpolation processing that does not depend on the estimation of an AC component signal such as cubic and linear, and an AC component estimation processing by the switching unit. is there.

First, the center-of-gravity alignment processing will be described.
When the image input is performed in a discrete system, if the number of pixels is different in the part where the correlation is calculated, the center of gravity of the color data is different from each other, causing a shift in the local region, and a false color is generated in a specific direction. Therefore, pixel interpolating means for matching the centers of gravity of the pixels of each spectral sensitivity characteristic in a predetermined area of the two input signals of the correlation coefficient calculating means is provided, and the data of the spectral sensitivity characteristic having a large amount of information is provided by the pixel interpolating means. The number is made equal to the number of data of the spectral sensitivity characteristic having a small amount of information. That is, the generation of the above-described false color can be suppressed by inputting each DC component to the correlation coefficient calculation unit after all the R, G, and B pixels have the same number of pixels by pixel interpolation.

The above will be described using a specific example. The input image data is composed of a luminance component Y as shown in FIG. 5A and two color difference components RY and BY as shown in FIGS. 5B and 5C. You. But Y and R
Since the data amount is different between -Y and BY, the position of the center of gravity is different. As shown in FIG.
The center of gravity of the upper left 2 × 2 pixel Y signal before interpolation is Y
(Above the black circle). In addition, FIG.
As shown in (C), the position of the center of gravity of the RY and BY signals is at the center of the 2 × 2 pixels (the black circle). The difference in the position of the center of gravity causes a directional false color to occur.

Therefore, Y, RY, B-
When all the Y pixels have the same number of pixels, as shown in FIGS. 5D, 5E, and 5F, Y, RY, B
The center of gravity of each signal of -Y is at the same position. In this way, it is possible to prevent the generation of a false color due to the difference in the center of gravity of each color signal.

Next, the block decomposition processing will be described.
Here, as shown in FIG. 6A, an image decomposing means for decomposing one image into a plurality of blocks (00 blocks to mn blocks) is provided, and the correlation coefficient and the Calculate similarity and calculate R, B
Is characterized by estimating the high-frequency component (AC component) of.

This will be specifically described below. FIG.
(C) shows one original image before processing. 2x
If the low-frequency component (DC component) and the high-frequency component (AC component) are defined as shown in FIG. 6B for each block of two pixels, the image shown in FIG. It will contain ingredients. As described above, when only a part of an image contains a high-frequency component, the following difference occurs between the case where one image is decomposed into a plurality of images and the case where no image is decomposed.

That is, when the estimation processing is performed without decomposing the image shown in FIG. 6C, the correlation coefficient (the ratio between the high-frequency component and the low-frequency component) in all the images is low. Since the difference of the frequency component depending on the position occurs, AC
The component is not properly estimated, and decreases after the processing as shown in FIG.

Thus, when one image as shown in FIG. 6E is decomposed into a plurality of blocks (here, 1 block = 2 × 2 pixels) and the estimation processing is performed, the image in the lower rightmost block is obtained. Correlation coefficient (ratio between high frequency component and low frequency component)
Is high, that is, since the difference of the frequency component depending on the position is reduced, the appropriate high-frequency component is estimated, and the high-frequency component does not decrease even after the estimation processing as shown in FIG.

According to the above-described block decomposition processing, even when the amount of high-frequency information depends on a part in one image, high-frequency components can be estimated without reducing the amount of high-frequency information.

Next, the switching process of the AC component signal estimation will be described. Here, the AC component signal of the photoelectric conversion region having the first spectral sensitivity characteristic and the second
AC component signal comparing means for comparing the AC component signal of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics, interpolation means that does not depend on estimation of the AC component such as cubic or linear, and comparison between the AC component signal comparing means. An AC component estimation method switching unit that adaptively switches between the AC component signal estimation process and the interpolation process such as cubic or linear according to the result is provided. That is, AC
The component signal comparing means (high-frequency information comparing means) compares the AC component information of each color signal. If the AC component information of G is smaller than the AC component information of R or B, the AC component signal estimation processing is replaced. And AC such as cubic and linear
Adaptive switching to interpolation processing not using component information generation is performed.

FIG. 7 shows how such an AC component signal estimation method is switched. That is, the correlation coefficient (the information amount of the AC component of the DCT coefficient of the second and subsequent spectral sensitivity characteristics and the AC information of the DCT coefficient of the first spectral sensitivity characteristic)
With respect to the point at which the ratio of the information amount of the component, that is, the ratio of the information amount of the R or B AC component to the information amount of the G AC component becomes 1, the left side where the correlation is small (AC component estimation) When the quantity <interpolation quantity>, interpolation processing such as cubic or linear is performed, and on the right side where the correlation is large (AC component estimation quantity> interpolation quantity), the AC component estimation processing (high-frequency estimation processing) is performed.

According to the above switching process, R or B
Artifacts (jagged noise when the color boundary is oblique) can be suppressed when the information on the AC component of G is smaller than the information on the AC component of (1).

A specific example of the AC component signal comparing means will be described below. In the first example, the AC component signal comparing means calculates the variance value of the AC component from the photoelectric conversion region having the first and second or later spectral sensitivity characteristics, and the obtained respective variance calculating means. And a comparing means for comparing the variance values. That is, as means for comparing the AC component information of each color signal, the AC
A means for comparing the dispersion of the components and the dispersion of the AC components of R and B is used.

That is, in the entire pixel array of n × m pixels forming the Bayer array as shown in FIG.

[0055]

(Equation 1)

Is used to determine the variance of the AC component for each of the colors R, G, and B. Next, the AC of the second and subsequent spectral sensitivity characteristics
The variance of the component and the variance of the AC component of the first spectral sensitivity characteristic (the ratio of the variance of the R or B AC component to the variance of the G AC component) are determined, and whether the value of this ratio is greater than 1 is determined. The above-described AC component estimation processing (high-frequency estimation processing) and interpolation processing are switched (FIG. 8B).

According to the first example described above, it is possible to suppress the occurrence of artifacts in an image in which G does not change and at least one of R and B changes. In addition, the amount of calculation is smaller than in the case where all the AC component information is compared, and it can be realized with a short processing time.

Hereinafter, a second example of the AC component signal comparing means will be described. In a second example, the AC component signal comparing means calculates the difference between the maximum value and the minimum value of the AC component from the photoelectric conversion regions having the first and second and subsequent spectral sensitivity characteristics, Comparing means for comparing the difference between the obtained maximum value and the minimum value. That is, a means for comparing the difference between the maximum value and the minimum value of G and the difference between the maximum value and the minimum value of R and B is used as means for comparing the information of the AC component.

More specifically, | the maximum value of the AC component of the DCT coefficient−the minimum value of the AC component of the DCT coefficient | of the second and subsequent spectral sensitivity characteristics, and | D of the first spectral sensitivity characteristic.
The ratio of the maximum value of the AC component of the CT coefficient−the minimum value of the AC component of the DCT coefficient | (here, the maximum value of the AC component of R or B−the minimum value of the AC component of R or B | and | G A
(The ratio of the maximum value of the C component−the minimum value of the AC component of G) |, the interpolation process is performed when the value of the ratio is smaller than 1, and the estimation process of the AC component is performed when the ratio value is larger than 1. (High-frequency estimation processing) is performed (FIG. 9).

In the second example, when the maximum AC component is generated only in a small area in one block as compared with the first example described above, an artifact is generated in another wide area in one block. However, the amount of calculation is small and the estimation process can be realized in a short processing time.

Hereinafter, a third example will be described. In the third example, the AC component signal comparing means compares the obtained variance value with a threshold value by the variance calculating means for calculating the variance value of the AC component from the photoelectric conversion region having the first spectral sensitivity characteristic. And a variance value comparing means.
That is, as a result of comparing the variances of the AC components of G, the information of the AC component of the G signal is lower than the threshold, and the high-frequency information (AC
If there is no significant difference in the pixel interpolation result between the case where the estimation of the component) is used and the case where the interpolation is performed by cubic or linear interpolation, adaptively to an interpolation means such as cubic or linear which does not generate the AC component information. Try to switch.

More specifically, the variance of the AC component of the second and subsequent spectral sensitivity characteristics and the AC component of the first spectral sensitivity characteristic
The variance of the component (the ratio of the variance of the R or B AC component to the variance of the G AC component) is determined, and the value of this ratio is a threshold (for example, 1.
1 or 1.2), the above A
Instead of C component estimation processing (high-frequency estimation processing), interpolation processing such as cubic or linear interpolation is switched (FIG. 10).

According to the above-described third example, the amount of calculation is smaller than that in the case where the threshold value is not used as in the first example, and substantially the same result can be obtained.

Hereinafter, a fourth example will be described. In a fourth example, the AC component signal comparing means calculates the difference between the maximum value and the minimum value of the AC component from the photoelectric conversion region having the first spectral sensitivity characteristic, A comparison means for comparing a difference between the maximum value and the minimum value with a threshold value is provided. If the information of the AC component of the G signal is lower than the threshold value and there is no significant difference in the pixel interpolation result between the case where the estimation of the AC component information is used and the case where interpolation such as cubic or linear is used, Adaptively switch to cubic or linear interpolation means that do not generate.

More specifically, | the maximum value of the AC component of the DCT coefficient−the minimum value of the AC component of the DCT coefficient | of the second and subsequent spectral sensitivity characteristics, and | D of the first spectral sensitivity characteristic.
The ratio of the maximum value of the AC component of the CT coefficient−the minimum value of the AC component of the DCT coefficient | (here, the maximum value of the AC component of R or B−the minimum value of the AC component of R or B | and | G A
The ratio between the maximum value of the C component−the minimum value of the AC component of G | is determined, and the value of this ratio is a threshold (for example, 1.1 or 1.2).
If it is smaller than the high frequency estimation process (AC
(Estimation of components) instead of interpolation processing such as cubic or linear (FIG. 11).

According to the above-described eighth embodiment, the amount of calculation is smaller than when no threshold is used as in the sixth embodiment, and substantially the same result can be obtained.

[0067]

According to the first aspect of the present invention, since the amount of calculation of high-frequency information is reduced, it is possible to provide a signal processing apparatus capable of obtaining an image equivalent to a conventional one in a short processing time. .

Further, according to the second aspect of the present invention, it is possible to provide a signal processing apparatus capable of suppressing the generation of a false color in a rapidly changing portion of data due to a difference in the position of the center of gravity of a local region.

Further, according to the third aspect of the present invention, it is possible to provide a signal processing device in which the case where the generation of the high frequency component is not performed is reduced although the estimation of the high frequency component is necessary. .

According to the fourth aspect of the present invention, the high-frequency information of a signal (for example, a G signal) in which the information amount of a signal related to at least one spectral sensitivity characteristic is larger than that of another spectral sensitivity characteristic is converted to another spectral sensitivity characteristic. A signal relating to the sensitivity characteristic (for example, R or B
Signal), the high-frequency information does not decrease, and the definition of an image originally having a signal (for example, an R or B signal) related to another spectral sensitivity characteristic does not deteriorate, and When cubic or linear interpolation is performed when the information amount is equal, a signal processing device that can obtain an equivalent image with a small amount of calculation can be provided.

According to the fifth aspect of the present invention, as compared with the case where all the high-frequency information is compared, almost the same judgment can be made, the signal amount can be reduced, and the signal can be realized in a short processing time. A processing device can be provided.

According to the sixth aspect of the present invention, A
It is possible to provide a signal processing device that can make almost the same determination as compared with the case where the variance of the C coefficient is used, has a small amount of calculation, and can be realized with a short processing time.

According to the seventh or eighth aspect of the present invention, it is possible to provide a signal processing device capable of realizing processing with a small amount of calculation when there is no large difference in output results. .

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a procedure of signal processing according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a procedure for performing high-frequency estimation when decompressing recorded or transmitted compressed image data.

FIG. 3 is a diagram illustrating a comparison between a conventional signal processing procedure and a signal processing procedure according to the present embodiment for simultaneously performing high-frequency component estimation and compression or decompression processing.

FIG. 4 is a diagram for explaining a relationship between a processing block when estimating a high-frequency component and a processing block when compressing / expanding data.

FIG. 5 is a diagram for explaining a center-of-gravity alignment process;

FIG. 6 is a diagram for describing a block decomposition process.

FIG. 7 illustrates switching processing of AC component signal estimation.

FIG. 8 is a diagram for describing a first example of AC component signal comparison means.

FIG. 9 is a diagram for explaining a second example of the AC component signal comparing means.

FIG. 10 is a diagram for explaining a third example of the AC component signal comparing means.

FIG. 11 is a diagram for describing a fourth example of the AC component signal comparing means.

FIG. 12 is a diagram for explaining a conventional signal processing procedure in which high-frequency estimation and image compression processing are separately performed.

FIG. 13 is a diagram for explaining a procedure for performing high-frequency estimation when decompressing recorded or transmitted compressed image data.

[Explanation of symbols]

 A11 Original signal input A12 Pixel interpolation A13 Storage in frame memory A14 Calculation of DCT coefficient A15 Calculation of correlation coefficient A16 Estimation of AC component A17 Compression using DCT coefficient A18 Recording or transmission

Continuation of the front page F term (reference) 5C057 AA06 DC01 EA01 EA02 EA07 EF05 EL01 EM09 GG01 GH03 5C059 KK01 KK11 MA23 MC32 MC34 PP15 PP16 SS14 TA21 TA31 TB08 TC04 TD02 TD04 TD05 TD12 UA02 UA33 5J003 A02 BC02 A02 BC

Claims (8)

[Claims] 1. A signal processing apparatus having image input means having a plurality of spectral sensitivity characteristics and processing a signal in which an information amount of a signal relating to at least one spectral sensitivity characteristic is larger than an information amount of a signal relating to another spectral sensitivity characteristic. In the plurality of spectral sensitivity characteristics, the number of photoelectric conversion unit regions having the first spectral sensitivity characteristic is a multiple of the number of photoelectric conversion unit regions having the second and subsequent spectral sensitivity characteristics. An existing image input means, a signal component of a photoelectric conversion area having the first spectral sensitivity characteristic input by the image input means, and a photoelectric conversion area having the second and subsequent spectral sensitivity characteristics having a small amount of information. DCT coefficient calculating means for calculating each DCT coefficient from the signal component; DCT coefficient of the photoelectric conversion region having the first spectral sensitivity characteristic obtained by the DCT coefficient calculating means A correlation coefficient calculating means for calculating a correlation coefficient between the DC component of the DCT coefficient and the DC component of the DCT coefficient of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information; The correlation coefficient obtained from the calculating means and the D
Based on the AC component of the DCT coefficient of the photoelectric conversion region having the first spectral sensitivity characteristic obtained by the CT coefficient calculating means, the DCT of the photoelectric conversion region having the second and subsequent spectral sensitivity characteristics having a small amount of information is obtained. AC that generates the AC component of the coefficient
A component signal generating unit; DC and AC components of DCT coefficients of a photoelectric conversion region having a first spectral sensitivity characteristic; and DC of DCT coefficients of a second and subsequent photoelectric conversion regions having a small amount of information.
Compression / expansion means for performing compression / expansion processing using the calculated DCT coefficient with respect to the AC component and the AC component. 2. The image processing apparatus according to claim 1, further comprising an interpolation unit configured to equalize the centers of gravity of pixels having respective spectral sensitivity characteristics in a predetermined region of the two DC component signals input to the correlation coefficient calculation unit. A signal processing device according to claim 1. 3. An image decomposing means for decomposing one image into a plurality of blocks, wherein the AC component signal generating means comprises:
2. The signal processing apparatus according to claim 1, wherein an AC component signal is generated for each block decomposed by the image decomposing unit. 4. An AC component signal of a DCT coefficient of a photoelectric conversion region having a first spectral sensitivity characteristic and a DCT of a second or later photoelectric conversion region having a small amount of information having a spectral sensitivity characteristic.
An AC component signal comparing unit that compares the AC component signal of the coefficient, an interpolation unit that does not depend on the estimation of the AC component signal such as cubic or linear, and a comparison result obtained by the AC component signal comparing unit.
The signal according to claim 1, further comprising: an AC component estimation method switching unit that adaptively switches between an AC component signal estimation process by the AC component signal generation unit and an interpolation process such as cubic or linear. Processing equipment. 5. The variance calculation means for calculating a variance value of an AC component signal of a DCT coefficient from a photoelectric conversion region having first and second and subsequent spectral sensitivity characteristics, and 5. The signal processing apparatus according to claim 4, further comprising a comparison unit configured to compare the obtained variance values. 6. The arithmetic unit for calculating a difference between a maximum value and a minimum value of an AC component signal of a DCT coefficient from a photoelectric conversion region having first and second and subsequent spectral sensitivity characteristics. 5. A signal processing apparatus according to claim 4, further comprising: means for comparing the obtained difference between the maximum value and the minimum value. 7. The variance calculating means for calculating the variance of the AC component signal of the DCT coefficient from the photoelectric conversion region having the first spectral sensitivity characteristic, and the obtained variance 5. The signal processing apparatus according to claim 4, further comprising a variance value comparison unit that compares the threshold value with a variance value. 8. The AC component signal comparing means, comprising: arithmetic means for calculating a difference between a maximum value and a minimum value of an AC component signal of a DCT coefficient from a photoelectric conversion region having a first spectral sensitivity characteristic; 5. A comparison means for comparing a difference between a maximum value and a minimum value of a component signal with a threshold value.
A signal processing device according to claim 1.