JP7442990B2 - Signal processing device and signal processing method - Google Patents

Description

The present invention relates to a signal processing device and a signal processing method.

Photoelectric conversion members used in digital cameras and the like have been developed using silicon as a single material. However, when silicon is used as a single material, the band gap is fixed and a color filter can only be formed on the surface. Therefore, in a photoelectric conversion member using silicon as a single material, color imaging cannot be performed with a single plate except by arranging red pixels, green pixels, and blue pixels in the plane direction. Hereinafter, red may be referred to as "R" or "Red," green as "G" or "Green," and blue as "B" or "Blue."

Furthermore, when capturing an image of a subject with higher resolution using the photoelectric conversion member as described above, brightness moire and color moire may occur. Therefore, in order to suppress the luminance moire and color moire, it is necessary to mount an optical low pass filter in the imaging device. However, when an optical low-pass filter is installed in an imaging device, the resolution decreases.

Therefore, Patent Document 1 discloses a photoelectric conversion member in which a silicon film having green pixels arranged therein is laminated on the incident light incident side of a silicon film having blue pixels arranged therein and red pixels arranged therein. In the photoelectric conversion member described in Patent Document 1, there are pixels that absorb green wavelengths on the entire surface of the photoelectric conversion member, so theoretically, luminance moire does not occur and color moire can be significantly suppressed. can.

However, in the photoelectric conversion member described in Patent Document 1, there are cases in which a pixel located in the upper layer that absorbs the Green wavelength also partially absorbs the Blue wavelength and the Red wavelength. Therefore, color reproducibility deteriorates, noise resistance in the blue region deteriorates, and color resolution also deteriorates. Therefore, Patent Document 2 describes that a green pixel signal is corrected (absorption correction) using at least one of a blue pixel signal and a red pixel signal.

Patent No. 4700947 US Patent Application Publication No. 2014/118579

However, in the photoelectric conversion member described in Patent Document 1, since Blue pixels and Red pixels are arranged in a checkerboard pattern, there are only about half as many Blue and Red pixels as there are Green pixels. . Therefore, Green's absorption correction becomes checkered. As a result, color reproducibility and color resolution deteriorate.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a signal processing device and a signal processing method that are excellent in color reproducibility and color resolution.

A signal processing device according to a first aspect of the present invention includes: a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light; a second photoelectric conversion layer that is provided on a side of the layer where incident light enters and photoelectrically converts third light having a wavelength different from the first light and the second light; a photoelectric conversion member comprising: A signal processing device for processing a signal generated by the first photoelectric conversion layer, wherein at least one of the first light signal and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer is processed by the first photoelectric conversion layer. an interpolation unit that performs interpolation using the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer; and an interpolation unit that interpolates the first light signal and the second light signal interpolated by the interpolation unit. and an absorption correction section that performs absorption correction of the third light signal using at least one of them.

According to the signal processing device according to the first aspect of the present invention, before performing absorption correction of the third light signal, at least one of the first light signal and the second light signal is transmitted to the third light signal. interpolated using the optical signal of Therefore, the third light signal can be subjected to absorption correction using at least one of the interpolated first light signal and second light signal. That is, it is possible to prevent the absorption correction of the third light signal from becoming checkered. Therefore, the signal processing device according to the first aspect of the present invention can perform signal processing with excellent color reproducibility and color resolution.

Further, the interpolation section uses intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer to calculate the intensity of the third light signal obtained by photoelectric conversion by the first photoelectric conversion layer. Preferably, at least one of the first light signal and the second light signal is interpolated.

Since the second photoelectric conversion layer partially absorbs at least one of the first light and the second light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes the first light and the second light. The second optical signal includes at least one of a part of the optical signal component and a part of the second optical signal component. Therefore, using the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer, the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer are Interpolation can be performed with higher accuracy by interpolating at least one of the optical signals.

Further, the absorption correction unit uses a correction value α determined in consideration of at least one of the first light component and the second light component absorbed by the second photoelectric conversion layer. , performing absorption correction of the third light signal by subtracting a value obtained by multiplying the first light signal value by α/(1−α) from the third light signal; is preferred.

Absorption correction of the third light signal using a correction value α determined in consideration of at least one of the first light component and the second light component partially absorbed by the second photoelectric conversion layer. By performing this, absorption correction can be performed with higher accuracy.

A signal processing device according to a second aspect of the present invention includes: a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light; a second photoelectric conversion layer that is provided on a side of the layer where incident light enters and photoelectrically converts third light having a wavelength different from the first light and the second light; a photoelectric conversion member comprising: A signal processing device for processing a signal generated by the first photoelectric conversion layer, wherein the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer is obtained by photoelectric conversion by the second photoelectric conversion layer. a first interpolation unit that performs interpolation using the third light signal; and absorption correction of the third light signal using the first light signal interpolated by the first interpolation unit. of the second light obtained by photoelectric conversion by the first photoelectric conversion layer using a third light signal absorption-corrected by the first absorption correction section; and a second interpolation unit that interpolates the signal.

According to the signal processing device according to the second aspect of the present invention, before performing absorption correction on the third light signal, the first light signal is interpolated using the third light signal. Therefore, the interpolated first light signal can be used to perform absorption correction on the third light signal. That is, it is possible to prevent the absorption correction of the third light signal from becoming checkered. Therefore, the signal processing device according to the second aspect of the present invention can perform signal processing with excellent color reproducibility and color resolution. In addition, since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes a component of the first light signal. Some of them are included. However, since the second light signal is interpolated using the absorption-corrected third light signal, the state in which the component of the first light signal included in the third light signal is removed Then, the second light signal can be interpolated. Thereby, it is possible to interpolate the second light signal with even higher accuracy.

Further, the first interpolation unit uses the intensity gradient information of the third light signal obtained by the photoelectric conversion by the second photoelectric conversion layer to obtain the signal obtained by the photoelectric conversion by the first photoelectric conversion layer. It is preferable to interpolate the first light signal.

Since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer contains a component of the first light signal. part included. Therefore, the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to interpolate the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer. By doing so, the first light signal can be interpolated with higher accuracy.

Further, the second interpolation section uses the intensity gradient information of the third light signal absorption-corrected by the first absorption correction section to calculate the Preferably, the second light signal is interpolated.

Since the second photoelectric conversion layer partially absorbs the second light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer contains a component of the second light signal. part included. Therefore, the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to interpolate the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer. By doing so, it is possible to interpolate the second light signal with higher accuracy.
In addition, since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes a component of the first light signal. Some of them are included. However, since the second light signal is interpolated using the intensity gradient information of the third light signal that has been absorption-corrected by the first absorption correction section, the The second light signal can be interpolated with the first light signal component removed. Thereby, it is possible to interpolate the second light signal with even higher accuracy.

Further, the first absorption correction unit corrects the first light signal by using a correction value α determined in consideration of a component of the first light absorbed by the second photoelectric conversion layer. Preferably, absorption correction of the third light signal is performed by subtracting a value obtained by multiplying the value by α/(1−α) from the third light signal.

By correcting the absorption of the third light signal using the correction value α determined in consideration of the component of the first light partially absorbed by the second photoelectric conversion layer, the third light signal is absorbed with higher precision. Corrections can be made.

Further, a second light signal that performs absorption correction of the third light signal that has been absorption-corrected by the first absorption correction section, using the second light signal interpolated by the second interpolation section. It is preferable to further include an absorption correction section.

Not only is absorption correction processing performed on the third light signal using the interpolated first light signal, but also absorption correction processing is further performed using the interpolated second light signal. conduct. Thereby, it is possible to generate a third light signal with even higher color reproducibility.

Further, in the signal processing device according to the first aspect and the second aspect of the present invention, the first photoelectric conversion layer emits blue light as the first light and red light as the second light. It is further preferable that light is photoelectrically converted, and the second photoelectric conversion layer photoelectrically converts green light as the third light.

Thereby, the second photoelectric conversion layer that absorbs green light can perform absorption correction of the green light signal using blue light, which is more easily absorbed. Therefore, signal processing with even better color reproducibility and color resolution can be performed.

A signal processing method according to a third aspect of the present invention includes: a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light; a second photoelectric conversion layer that is provided on a side of the layer where incident light enters and photoelectrically converts third light having a wavelength different from the first light and the second light; a photoelectric conversion member comprising: A signal processing method for processing a signal generated by the first photoelectric conversion layer, wherein at least one of the first light signal and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer is processed by the first photoelectric conversion layer. an interpolation step of interpolating using the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer; and an interpolation step of interpolating the first light signal and the second light signal interpolated by the interpolation unit. and an absorption correction step of performing absorption correction of the third light signal using at least one of the signals.

In the signal processing method according to the third aspect of the present invention, before performing absorption correction of the third light signal, at least one of the first light signal and the second light signal is transmitted to the third light signal. interpolated using the signal of Therefore, the third light signal can be subjected to absorption correction using at least one of the interpolated first light signal and second light signal. That is, it is possible to prevent the absorption correction of the third light signal from becoming checkered. Therefore, according to the signal processing method according to the third aspect of the present invention, signal processing with excellent color reproducibility and color resolution can be performed.

Further, in the interpolation step, intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to calculate the intensity gradient information of the third light signal obtained by photoelectric conversion by the first photoelectric conversion layer. Preferably, at least one of the first light signal and the second light signal is interpolated.

Since the second photoelectric conversion layer partially absorbs at least one of the first light and the second light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes the first light and the second light. The second optical signal includes at least one of a part of the optical signal component and a part of the second optical signal component. Therefore, using the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer, the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer are Interpolation can be performed with higher accuracy by interpolating at least one of the optical signals.

In the absorption correction step, using a correction value α determined in consideration of at least one of the first light component and the second light component absorbed by the second photoelectric conversion layer, Preferably, absorption correction of the third light signal is performed by subtracting a value obtained by multiplying the first light signal value by α/(1-α) from the third light signal. .

Absorption correction of the third light signal using a correction value α determined in consideration of at least one of the first light component and the second light component partially absorbed by the second photoelectric conversion layer. By performing this, absorption correction can be performed with higher accuracy.

A signal processing method according to a fourth aspect of the present invention includes: a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light; a second photoelectric conversion layer that is provided on a side of the layer where incident light enters and that photoelectrically converts third light having a different wavelength from the first light and the second light; a photoelectric conversion member comprising: A signal processing method for processing a signal generated by the method, wherein the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer is obtained by photoelectric conversion by the second photoelectric conversion layer. a first interpolation step of interpolating using the third light signal; and absorption correction of the third light signal using the first light signal interpolated in the first interpolation step. of the second light obtained by photoelectric conversion by the first photoelectric conversion layer using the third light signal absorption-corrected in the first absorption correction step. a second interpolation step of interpolating the signal.

In the signal processing method according to the fourth aspect of the present invention, the first light signal is interpolated using the third light signal before performing absorption correction on the third light signal. Therefore, the interpolated first light signal can be used to perform absorption correction on the third light signal. That is, it is possible to prevent the absorption correction of the third light signal from becoming checkered. Therefore, according to the signal processing method according to the fourth aspect of the present invention, signal processing with excellent color reproducibility and color resolution can be performed. In addition, since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes a component of the first light signal. is included in part. However, since the second light signal is interpolated using the absorption-corrected third light signal, the state in which the component of the first light signal included in the third light signal is removed Then, the second light signal can be interpolated. Thereby, it is possible to interpolate the second light signal with even higher accuracy.

In the first interpolation step, intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to obtain the signal obtained by photoelectric conversion by the first photoelectric conversion layer. Preferably, the first light signal is interpolated.

Since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer contains a component of the first light signal. part included. Therefore, the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to interpolate the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer. By doing so, the first light signal can be interpolated with higher accuracy.

In the second interpolation step, intensity gradient information of the third light signal absorption-corrected in the first absorption correction step is used to calculate the second It is preferable to interpolate the optical signal of .

Since the second photoelectric conversion layer partially absorbs the second light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer contains a component of the second light signal. part included. Therefore, the intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to interpolate the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer. By doing so, it is possible to interpolate the second light signal with higher accuracy.
In addition, since the second photoelectric conversion layer partially absorbs the first light, the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer includes a component of the first light signal. Some of them are included. However, since the second light signal is interpolated using the intensity gradient information of the third light signal that has undergone absorption correction in the first absorption correction step, the The second light signal can be interpolated with the first light signal component removed. Thereby, it is possible to interpolate the second light signal with even higher accuracy.

In the first absorption correction step, the signal value of the first light is adjusted using a correction value α determined in consideration of the component of the first light absorbed by the second photoelectric conversion layer. Preferably, absorption correction of the third light signal is performed by subtracting a value obtained by multiplying α/(1−α) from the third light signal.

By correcting the absorption of the third light signal using the correction value α determined in consideration of the component of the first light partially absorbed by the second photoelectric conversion layer, the third light signal is absorbed with higher precision. Corrections can be made.

Further, a second method for performing absorption correction of the third light signal subjected to absorption correction in the first absorption correction step using the second light signal interpolated in the second interpolation step. Preferably, the method further includes an absorption correction step.

Not only is absorption correction processing performed on the third light signal using the interpolated first light signal, but also absorption correction processing is further performed using the interpolated second light signal. conduct. Thereby, it is possible to generate a third light signal with even higher color reproducibility.

Further, in the signal processing method according to the third aspect and the fourth aspect of the present invention, the first photoelectric conversion layer emits blue light as the first light and red light as the second light. It is further preferable that light is photoelectrically converted, and the second photoelectric conversion layer photoelectrically converts green light as the third light.

Thereby, the second photoelectric conversion layer that absorbs green light can perform absorption correction of the green light signal using blue light, which is more easily absorbed. Therefore, signal processing with even better color reproducibility and color resolution can be performed.

According to the present invention, it is possible to provide a signal processing device and a signal processing method with excellent color reproducibility and color resolution.

1 is a schematic diagram showing an example of a photoelectric conversion member according to Embodiment 1. FIG. 1 is a diagram showing an example of a signal processing device according to Embodiment 1. FIG. 1 is a diagram illustrating an example of a signal processing method according to Embodiment 1. FIG. 3 is a diagram illustrating an example of interpolation processing in the signal processing device according to Embodiment 1. FIG. FIG. 3 is a diagram showing an example of an interpolation circuit in the signal processing device according to the first embodiment. FIG. 3 is a diagram showing an example of the result of absorption correction in the signal processing device according to the first embodiment. 3 is a diagram illustrating an example of a signal processing device according to a second embodiment. FIG. 7 is a diagram illustrating an example of a signal processing method according to Embodiment 2. FIG. 8 is a diagram conceptually showing the interpolation process shown in C of FIG. 8, the absorption correction process shown in D of FIG. 8, and the interpolation process shown in E of FIG. 8. 1 is a diagram showing an example of a conventional signal processing method. 2 is a diagram showing an image output by the signal processing method of Example 1 and an image output by the signal processing method of Comparative Example 1. FIG. 2 is a diagram showing an image output by the signal processing method of Example 1 and an image output by the signal processing method of Comparative Example 1. FIG. 2 is a diagram showing an image output by the signal processing method of Example 1 and an image output by the signal processing method of Example 2. FIG. FIG. 3 is a diagram illustrating an example of a signal processing device according to a third embodiment. 7 is a diagram illustrating an example of a signal processing method according to Embodiment 3. FIG. 15 is a diagram conceptually showing the interpolation process shown in C of FIG. 15, the absorption correction process shown in D of FIG. 15, the interpolation process shown in E of FIG. 15, and the absorption correction process shown in F of FIG. 15. 7 is a diagram illustrating an example of the result of Red absorption correction in the signal processing device according to Embodiment 3. FIG.

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic diagram showing an example of a photoelectric conversion member according to an embodiment of the present invention.
A signal processing device is a device that processes a signal generated by imaging in an imaging device such as a digital camera, that is, a signal representing a captured image. Further, the signal processing device may include an imaging section of an imaging device. The imaging unit may include an image sensor as a photoelectric conversion member shown in FIG. Further, the imaging section may include a lens of an optical system. The optical system lens forms an image of the light emitted from the subject on the light receiving surface of the photoelectric conversion member, and the photoelectric conversion member photoelectrically converts the received light to generate a signal indicating the captured image. That is, the signal processing device may be an imaging device that includes an imaging section that includes a photoelectric conversion member, an optical lens, and the like.

Examples of signals generated by imaging in an imaging device include a red light signal, a green light signal, and a blue light signal. Further, the signals generated by imaging in the imaging device include, for example, a signal obtained as a result of photoelectric conversion of light with a NIR (near-infrared) wavelength, that is, a signal indicating a so-called infrared image. Good too.

As shown in FIG. 1, the photoelectric conversion member includes, for example, a first photoelectric conversion layer L1 and a second photoelectric conversion layer L2. A red light signal as the second light and a green light signal as the third light are generated.

The first photoelectric conversion layer L1 is a photoelectric conversion layer that photoelectrically converts blue light (first light) and red light (second light). Further, the second photoelectric conversion layer L2 is a photoelectric conversion layer that photoelectrically converts green light (third light). Moreover, the second photoelectric conversion layer L2 is provided on the side where incident light enters rather than the first photoelectric conversion layer L1. When the direction in which incident light enters is defined as "upward", the second photoelectric conversion layer L2 corresponds to the upper layer, and the first photoelectric conversion layer L1 corresponds to the lower layer. In other words, the photoelectric conversion member is a stacked-type imaging device in which a plurality of photoelectric conversion layers are stacked.

A transparent electrode made of, for example, ITO (Indium Tin Oxide) is provided between the first photoelectric conversion layer L1 and the second photoelectric conversion layer L2. Examples of the first photoelectric conversion layer L1 include a substrate made of silicon and provided with a photoelectric conversion element. The transparent electrode is electrically connected to a circuit formed on the substrate. Further, as the second photoelectric conversion layer L2, for example, an organic film can be used.

In the example shown in FIG. 1, the first photoelectric conversion layer L1 is formed by forming R pixels and B pixels on a substrate using a conventional semiconductor process using silicon. Further, the R pixels and the B pixels are arranged in a checkered pattern, for example. Note that the arrangement of R pixels and B pixels is not limited to the checkered pattern.

In addition, in the example shown in FIG. 1, an organic photoelectric film that absorbs light of wavelength G is formed on the upper layer of the substrate on which B pixels and R pixels are formed, thereby forming a second photoelectric conversion layer. L2 is formed.

In the configuration shown in FIG. 1, since there are as many G pixels as all the pixels of the photoelectric conversion member, theoretically, brightness moire does not occur, and color moire is also significantly suppressed.

However, in the photoelectric conversion member shown in FIG. 1, when a material with a high external quantum efficiency of G is used as the second photoelectric conversion layer L2, only the light of the wavelength G is used in the second photoelectric conversion layer L2. However, some of the light with the B wavelength or the R wavelength is also absorbed, resulting in photoelectric conversion. Therefore, if the signals photoelectrically converted in the first photoelectric conversion layer L1 and the second photoelectric conversion layer L2 are used as they are and subjected to color correction processing and output, color reproducibility may deteriorate, Problems such as deterioration of noise resistance and deterioration of resolution occur.

Note that the example of the first photoelectric conversion layer L1 and the second photoelectric conversion layer L2 is not limited to the example shown in FIG. 1.
For example, in the first photoelectric conversion layer L1, an R pixel, a G pixel, and a B pixel may be formed on a substrate by a conventional semiconductor process using silicon. In this case, R pixels, G pixels, and B pixels are arranged in, for example, a Bayer array.
Further, the second photoelectric conversion layer L2 may be an NIR layer that absorbs light with a NIR wavelength and performs photoelectric conversion.
Even if the first photoelectric conversion layer L1 and the second photoelectric conversion layer L2 have other configurations shown above, the same problem as the configuration shown in FIG. 1 may occur.

Therefore, a signal processing device according to an embodiment of the present invention performs absorption correction processing on a signal generated by imaging in an imaging device. The absorption correction process is performed in an absorption correction section included in the signal processing device. Specifically, the signal processing device detects at least one component of blue light and red light that is absorbed in the second photoelectric conversion layer L2 and should be photoelectrically converted in the first photoelectric conversion layer L1. The green light signal obtained by photoelectric conversion by the second photoelectric conversion layer L2 is corrected. More specifically, the signal processing device uses at least one of a blue light signal and a red light signal obtained by photoelectric conversion by the first photoelectric conversion layer L1, and the signal processing by the second photoelectric conversion layer L2. Performs absorption correction for green light signals obtained by photoelectric conversion. This can reduce the influence of light components of wavelengths that are absorbed in the upper layer and photoelectrically converted in the lower layer. An example of the absorption correction process will be described later.

Further, as shown in FIG. 1, in the second photoelectric conversion layer L2, the G pixels exist over all the pixels of the photoelectric conversion member, whereas in the first photoelectric conversion layer L1, the R pixels exist. The pixels of and B do not exist over all the pixels of the photoelectric conversion member. Therefore, by using at least one of the blue light signal and the red light signal obtained by photoelectric conversion by the first photoelectric conversion layer L1, the green light signal obtained by photoelectric conversion by the second photoelectric conversion layer L2 is used. When the signal is subjected to absorption correction, the green light signal is subjected to absorption correction in the form of an arrangement pattern of R pixels or B pixels. As a result, color reproducibility and color resolution deteriorate.

Therefore, the signal processing device according to the present embodiment performs the photoelectric conversion by the first photoelectric conversion layer L1 before performing the absorption correction process of the green light signal obtained by the photoelectric conversion by the second photoelectric conversion layer L2. interpolate at least one of the blue light signal and the red light signal obtained by the method. The interpolation process is performed, for example, in an interpolation unit, a first interpolation unit, and a second interpolation unit included in the signal processing device. Specifically, the signal processing device uses a green light signal obtained by photoelectric conversion by the second photoelectric conversion layer L2 and a blue light signal obtained by photoelectric conversion by the first photoelectric conversion layer L1. and interpolating at least one of the red light signal. As a result, the green light signal can be absorbed and corrected using at least one of the interpolated blue light signal and red light signal, and the green light signal is It is possible to prevent absorption correction from occurring in the pixel arrangement pattern. An example of the interpolation process will be described later.

Further, the signal processing device according to the present embodiment performs absorption correction processing on a signal generated by imaging in an imaging device, and then performs color correction processing. Therefore, the signal processing device is affected by deterioration of color reproducibility, deterioration of noise resistance in the B region or R region, deterioration of resolution, etc. that may occur when using a photoelectric conversion member in which multiple photoelectric conversion layers are laminated. can be reduced. Therefore, the signal processing device can improve the quality of captured images. Examples of color correction processing include processing for finely adjusting colors by matrix calculation using a color correction matrix. Color correction processing is performed in a color correction section of the signal processing device. An example of the color correction process will be described later. Note that the signal processing device according to this embodiment may further perform white balance processing.

The signal processing device includes a CPU (Central Processing Unit) (not shown), a storage unit (not shown), and the like. All processing in the signal processing device is realized by the CPU executing the program stored in the storage unit. For example, the interpolation process, the absorption correction process, the color correction process, etc. are realized by the CPU executing a program stored in the storage unit. That is, the CPU functions as an interpolation section, an absorption correction section, a first interpolation section, a second interpolation section, and a color correction section of the present invention by executing a program stored in the storage section.
Furthermore, the interpolation processing, absorption correction processing, and color correction processing in the signal processing device may be realized by an image processing circuit or the like included in the signal processing device.

Embodiment 1
Hereinafter, the signal processing device 100 and signal processing method according to Embodiment 1 of the present invention will be described in more detail. FIG. 2 is a diagram illustrating an example of the signal processing device 100 according to the first embodiment. Further, FIG. 3 is a diagram illustrating an example of the signal processing method according to the first embodiment.

Below, the case where the 1st photoelectric conversion layer L1 and the 2nd photoelectric conversion layer L2 have the structure shown in FIG. 1 is mentioned as an example. In addition, in the following, the signal processing device 100 is configured to perform processing for each of the green light signal and the blue light signal, or for the green light signal based on the green light signal and the blue light signal. On the other hand, an example will be exemplified in which absorption correction processing is performed. Note that the signal processing device 100 performs processing for each of the green light signal and the red light signal, or for the green light signal, based on the green light signal and the red light signal. Absorption correction processing may also be performed. Further, the signal processing device 100 generates a green light signal, a red light signal, and a blue light signal based on the green light signal, the red light signal, and the blue light signal. Absorption correction processing may be performed on each of them.

As shown in FIG. 2, the signal processing device 100 according to the first embodiment includes a Blue/Red holding line memory (LM) 152, a Green holding line memory (LM) 154, and a write counter (Write counter). 156, a read counter 158, a Blue/Red interpolation circuit 160 as an interpolation section, a sub-absorption correction circuit 164 as an absorption correction section, a color correction matrix circuit (not shown), and the like.

The Blue/Red holding line memory 152 is a line memory that holds a blue light signal Bin and a red light signal Rin input from the photoelectric conversion member, respectively. Further, the Green holding line memory 154 is a line memory that holds the green light signal Gin input from the photoelectric conversion member.

Writing to each of the Blue/Red holding line memory 152 and Green holding line memory 154 is controlled by a write counter 156. Further, reading from each of the Blue/Red holding line memory 152 and the Green holding line memory 154 is controlled by a read counter 158.

The write counter 156 and the read counter 158 perform counting based on, for example, a change in the signal level of a synchronization signal transmitted from a CPU (not shown) or the like that constitutes the signal processing device 100 according to the first embodiment. Each of the write counter 156 and the read counter 158 controls the Blue/Red holding line memory 152 and the Green holding line memory 154, respectively, according to the count value. Note that the method of controlling each of the write counter 156 and the read counter 158 is not particularly limited as long as the processing shown in FIG. 3 can be realized.

The Blue/Red interpolation circuit 160 performs interpolation on the blue light signal Bin and the red light signal Rin read out from the Blue/Red holding line memory 152. Specifically, the Blue/Red interpolation circuit 160 refers to the gradient (intensity gradient information) of the pixel value of the green pixel indicated by the green light signal Gin read out from the Green holding line memory 154. Interpolation is performed on the light signal Bin and the red light signal Rin.

The sub-absorption correction circuit 164 performs absorption correction processing on the green light signal Gin based on the green light signal Gin read out from the Green holding line memory 154 and the interpolated blue light signal Bo. conduct. Note that the sub-absorption correction circuit 164 may also perform absorption correction processing on the interpolated blue light signal Bo.

Then, the blue light signal Bo and the red light signal Ro interpolated by the Blue/Red interpolation circuit 160 are input to a color correction matrix circuit (not shown), and are subjected to absorption correction processing from the sub-absorption correction circuit 164. The green light signal Go thus obtained is input to a color correction matrix circuit (not shown). Then, the color correction matrix circuit performs color correction processing on the blue light signal Bo, the red light signal Ro, and the green light signal Go.

As shown in FIG. 3, the signal processing method according to the first embodiment includes a Blue/Red interpolation process as an interpolation step shown in C of FIG. 3, a Blue absorption correction process as an absorption correction step shown in D of FIG. A color correction matrix process as a color correction step shown in FIG. 3E is provided.

A blue light signal Bin, a green light signal Gin, and a red light signal Rin shown in FIG. 3A are input to the signal processing device 100 from the photoelectric conversion member shown in FIG. 3B. Then, the Blue/Red interpolation circuit 160 of the signal processing device 100 performs Blue/Red interpolation processing shown in C in FIG. 3 on the blue light signal Bin and the red light signal Rin.
Next, the sub-absorption correction circuit 164 of the signal processing device 100 uses the blue light signal Bo that has been subjected to the Blue/Red interpolation process to adjust the blue absorption correction shown in D in FIG. 3 to the green light signal Gin. Make corrections. In addition, in the Blue absorption correction process shown in D of FIG. 3, the absorption correction process may also be performed on the interpolated blue light signal Bo.
Next, the color correction matrix circuit of the signal processing device 100 performs the color correction matrix processing shown in E in FIG. 3 on the blue light signal Bo, the red light signal Ro, and the green light signal Go. I do.

Next, an example of the interpolation process according to the first embodiment will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram illustrating an example of Blue/Red interpolation processing in the signal processing device 100. The upper side of FIG. 4 shows a block of 3 pixels x 3 pixels in the first photoelectric conversion layer L1, and the lower side of FIG. 4 shows the second photoelectric conversion layer corresponding to the block of 3 pixels x 3 pixels. A block of 3 pixels x 3 pixels in layer L2 is shown. FIG. 5 is a diagram showing an example of the Blue/Red interpolation circuit 160 in the signal processing device 100. As shown in FIG. 5, the Blue/Red interpolation circuit 160 includes a Green interpixel correlation information extraction section 160A, an interpolation ratio calculation section 160B, and the like.

As shown in FIG. 4, in the first photoelectric conversion layer L1, R pixels and B pixels are arranged in a checkered pattern. In the 3-pixel×3-pixel block shown in the upper part of FIG. 4, the center pixel (target pixel) is an R pixel, and the blue light signal Bin does not have spatial resolution information in the center pixel. Hereinafter, a case will be described as an example in which a center pixel (target pixel) is interpolated using surrounding pixels B1, B2, B3, and B4 in the 3-pixel×3-pixel block. The center pixel (target pixel) to be interpolated is assumed to be B0. Note that the Blue/Red interpolation circuit 160 also performs interpolation processing on the red light signal Rin using a method similar to the method described below.

The Green inter-pixel correlation information extraction unit 160A extracts correlation information between pixels of the green light signal Gin. For example, the Green inter-pixel correlation information extraction unit 160A extracts information between G0, G1, G2, G3, and G4 corresponding to the center pixel B0, pixels B1, B2, B3, and B4 in the 3-pixel×3-pixel block shown in FIG. The gradient information (intensity gradient information) of the pixel value is extracted. Specifically, the Green inter-pixel correlation information extraction unit 160A extracts gradient information G1s of pixel values between the pixel G0 and the pixels G1, G2, G3, and G4 according to the following equations (1) to (4), Calculate G2s, G3s, and G4s.
G1s=abs(G1-G0)...(1)
G2s=abs(G2-G0)...(2)
G3s=abs(G3-G0)...(3)
G4s=abs(G4-G0)...(4)

The interpolation ratio calculation unit 160B compares the gradient information G1s, G2s, G3s, and G4s of the pixel values between the pixel G0 and the pixels G1, G2, G3, and G4 calculated by the Green inter-pixel correlation information extraction unit 160A, and calculates the highest Calculate the gradient of small pixel values as an interpolation ratio. Then, the interpolation ratio calculation unit 160B interpolates the center pixel B0 of the block shown in the upper side of FIG. 4 using the gradient of the smallest pixel value (interpolation ratio). Specifically, the interpolation ratio calculation unit 160B calculates the interpolation ratio according to the following equations (5) to (8), and interpolates the center pixel of the block shown in the upper side of FIG. 4.
if(G1s==min(G1s, G2s, G3s, G4s))
B0=(B1×G1)/G0 (5)
if(G2s==min(G1s, G2s, G3s, G4s))
B0=(B2×G2)/G0 (6)
if(G3s==min(G1s, G2s, G3s, G4s))
B0=(B3×G3)/G0 (7)
if(G4s==min(G1s, G2s, G3s, G4s))
B0=(B4×G4)/G0 (8)

Next, an example of absorption correction processing according to the first embodiment will be described.
The sub-absorption correction circuit 164 performs Blue absorption correction on the green light signal Gin using the blue light signal Bo that has been subjected to the Blue/Red interpolation process. Specifically, the sub-absorption correction circuit 164 determines the correction value α in consideration of the blue light component absorbed by the second photoelectric conversion layer L2. Next, the sub-absorption correction circuit 164 subtracts the value obtained by multiplying the value (signal value) of the blue light signal Bin by α/(1-α) from the green light signal Gin. Blue absorption correction is performed on the light signal Gin.
FIG. 6 shows an example of the results of absorption correction in the signal processing device 100 according to the first embodiment. The left side of FIG. 6 shows the intensity distribution of the red light signal Rin, the green light signal Gin, and the blue light signal Bin, which are input to the signal processing device 100 from the photoelectric conversion member. In addition, on the right side of FIG. 6, the intensity distributions of the interpolated red light signal Ro, the absorption-corrected green light signal Go, and the interpolated and absorption-corrected blue light signal Bo are shown. It shows.

In addition, examples of the correction value α include the following examples. Note that the correction value α is not limited to the example shown below.
Example 1: Minimum value Gbottom of the green light signal Gin generated by the second photoelectric conversion layer L2 in the blue region
Example 2: Absorption rate Gabs of green light absorbed by the second photoelectric conversion layer L2 at the wavelength of the peak value of the blue light signal Bin generated by the first photoelectric conversion layer L1

When the correction value α is Gbottom shown in Example 1 above, the sub-absorption correction circuit 164 calculates Gbottom as the correction value α according to the following equation (9).
Go=Gin-Gbottom...(9)

When the correction value α is Gabs shown in Example 2 above, the sub-absorption correction circuit 164 calculates Gabs as the correction value α according to the following equation (10).
Go=Gin-Gabs...(10)

Further, the sub-absorption correction circuit 164 may perform Blue absorption correction on the interpolated blue light signal Bo according to the following equation (11) or equation (12).
Bo'=Bo+Gbottom...(11)
Bo'=Bo+Gabs...(12)

Note that, similarly to the above, the sub absorption correction circuit 164 may perform Red absorption correction on the green light signal Gin using the red light signal Ro that has been subjected to the Blue/Red interpolation process. Specifically, the sub-absorption correction circuit 164 determines the correction value α' in consideration of the red light component absorbed by the second photoelectric conversion layer L2. Next, the sub-absorption correction circuit 164 subtracts the value obtained by multiplying the value of the red light signal Rin by α'/(1-α') from the green light signal Gin. Red absorption correction is performed on the signal Gin.

In addition, examples of the correction value α' include the following examples. Note that the correction value α' is not limited to the example shown below.
Example 1: Minimum value Gbottom' of the green light signal Gin generated by the second photoelectric conversion layer L2 in the red region
Example 2: Absorption rate Gabs' of green light absorbed by the second photoelectric conversion layer L2 at the wavelength of the peak value of the red light signal Rin generated by the first photoelectric conversion layer L1

When the correction value α' is Gbottom' shown in Example 1 above, the sub-absorption correction circuit 164 calculates Gbottom' as the correction value α' according to the following equation (13).
Go=Gin-Gbottom'...(13)

When the correction value α' is Gabs' shown in Example 2 above, the sub-absorption correction circuit 164 calculates Gabs' as the correction value α' according to the following equation (14).
Go=Gin-Gabs'...(14)

Further, the sub absorption correction circuit 164 may perform Red absorption correction on the interpolated red light signal Ro according to the following equation (15) or equation (16).
Ro'=Ro+Gbottom' (15)
Ro'=Ro+Gabs'...(16)

Next, an example of color correction processing according to the first embodiment will be described.
A color correction matrix circuit (not shown) of the signal processing device 100 outputs an interpolated blue light signal Bo, an interpolated red light signal Ro, and an absorption-corrected green light signal Go. Then, color correction processing is performed. Note that the color correction matrix circuit may perform color correction processing on the interpolated and absorption-corrected blue light signal Bo' instead of the interpolated blue light signal Bo. Further, the color correction matrix circuit may perform color correction processing on the interpolated and absorption-corrected red light signal Ro' instead of the interpolated red light signal Ro.

Specifically, the color correction matrix circuit performs color correction processing according to the following equation (17). That is, the color correction matrix circuit performs color correction processing by performing matrix operations using a color correction matrix that is a 3×3 matrix as shown in Equation (17) below. In Equation (17) below, Rin is the interpolated red light signal Ro or the interpolated and absorption-corrected red light signal Ro', and Gin is the absorption-corrected red light signal Ro'. Bin is the interpolated blue light signal Bo or the interpolated and absorption-corrected blue light signal Bo'. Note that the color correction processing in the first embodiment is not limited to matrix calculation using a color correction matrix.

In the signal processing device 100 and signal processing method according to the first embodiment described above, before performing absorption correction of the third light signal, the blue light signal Bin and the red light signal Rin are At least one of them is interpolated using the green light signal Gin. Therefore, the green light signal Gin can be subjected to absorption correction using at least one of the interpolated blue light signal Bo and red light signal Ro. That is, it is possible to prevent the absorption correction of the green light signal Gin from becoming checkered. Therefore, the signal processing device 100 and the signal processing method according to the first aspect of the present invention can perform signal processing with excellent color reproducibility and color resolution.

In addition, since the second photoelectric conversion layer L2 partially absorbs blue light and red light, the green light signal Gin obtained by photoelectric conversion by the second photoelectric conversion layer L2 includes blue light. It contains a part of the signal component of the red light signal and a part of the red light signal component. Therefore, using the gradient information (intensity gradient information) of the pixel value of the green light signal Gin obtained by photoelectric conversion by the second photoelectric conversion layer L2, the signal Gin obtained by photoelectric conversion by the first photoelectric conversion layer L1 is used. By interpolating the blue light signal Bin and the red light signal Rin, interpolation can be performed with higher accuracy.

In addition, the correction value α determined in consideration of the blue light component partially absorbed by the second photoelectric conversion layer L2, or the correction value α determined considering the component of the blue light partially absorbed by the second photoelectric conversion layer L2, By performing absorption correction on the green light signal Gin using the correction value α' determined in consideration of the components, absorption correction can be performed with higher accuracy.

Further, the absorption correction of the interpolated blue light signal Bo is performed using the correction value α determined in consideration of the blue light component partially absorbed by the second photoelectric conversion layer L2. Therefore, absorption correction can be performed with even higher precision. In addition, the absorption correction of the interpolated red light signal Ro is performed using the correction value α' determined in consideration of the red light component partially absorbed by the second photoelectric conversion layer L2. By doing so, absorption correction can be performed with even higher accuracy.

Embodiment 2
Next, a second embodiment of the present invention will be described with reference to FIGS. 7 to 9. FIG. 7 is a diagram illustrating an example of a signal processing device 200 according to the second embodiment. FIG. 8 is a diagram illustrating an example of a signal processing method according to the second embodiment. FIG. 9 is a diagram conceptually showing the interpolation process shown in C of FIG. 8, the absorption correction process shown in D of FIG. 8, and the interpolation process shown in E of FIG. 8.
As shown in FIG. 7, the signal processing device 200 according to the second embodiment does not include a Blue/Red interpolation circuit 160, a Blue/Red delay line memory (LM; Line Memory) 162, and a sub absorption correction circuit 164. A blue interpolation circuit 166 as a first interpolation section, a blue delay line memory (LM) 168, a blue sub-absorption correction circuit 170 as a first absorption correction section, a green delay line memory (LM) 172, This signal processing device 100 differs from the signal processing device 100 according to the first embodiment in that it includes a Red interpolation circuit 174 as a second interpolation section. Hereinafter, in the signal processing device 200 according to the second embodiment, the same components as those in the signal processing device 100 according to the first embodiment will be denoted by the same reference numerals, and the description thereof will be omitted.

The Blue interpolation circuit 166 performs interpolation on the blue light signal Bin read from the Blue/Red holding line memory 152. The Blue interpolation circuit 166 performs interpolation on the blue light signal Bin while referring to the gradient (intensity gradient information) of the pixel value of the green pixel indicated by the green light signal Gin read out from the Green holding line memory 154. I do.

The Blue delay line memory 168 is a line memory that holds the interpolated blue light signal Bo obtained as a result of processing in the Blue interpolation circuit 166. The Blue delay line memory 168 functions, for example, as a delay element that delays the output of the interpolated blue light signal Bo by a time synchronized with the output of the green light signal or the output of the red light signal.

The Blue sub-absorption correction circuit 170 performs absorption correction processing on the green light signal Gin based on the green light signal Gin read out from the Green holding line memory 154 and the interpolated blue light signal Bo. I do. Note that the blue sub-absorption correction circuit 170 may perform absorption correction processing on the interpolated blue light signal Bo.

The Green delay line memory 172 is a line memory that holds the green light signal Go obtained as a result of processing in the Blue sub-absorption correction circuit 170. The Green delay line memory 172 functions as a delay element that delays the output of the green light signal Go by a time synchronized with the output of the blue light signal Bo.

The Red interpolation circuit 174 performs interpolation on the red light signal Rin read out from the Blue/Red holding line memory 152. The Red interpolation circuit 174 performs interpolation on the red light signal Rin while referring to the gradient (intensity gradient information) of the pixel value of the green pixel indicated by the green light signal Go read out from the Green delay line memory 172. I do.

The blue light signal Bo subjected to interpolation processing in the Blue interpolation circuit 166 is input to a color correction matrix circuit (not shown) via the Blue delay line memory 168, and the blue light signal Bo subjected to absorption correction processing is input from the Blue sub-absorption correction circuit 170. The light signal Go is input to a color correction matrix circuit (not shown), and the red light signal Ro interpolated by the Red interpolation circuit 174 is input to a color correction matrix circuit (not shown). Then, the color correction matrix circuit performs color correction processing on the blue light signal Bo, the red light signal Ro, and the green light signal Go.

As shown in FIG. 8, the signal processing method according to the second embodiment includes a blue interpolation process as a first interpolation step shown in C of FIG. 8, and a blue interpolation process as a first absorption correction step shown in D of FIG. It includes an absorption correction process, a red interpolation process as a second interpolation step shown in E of FIG. 8, and a color correction matrix process as a color correction step shown in F of FIG.
A blue light signal Bin, a green light signal Gin, and a red light signal Rin shown in FIG. 8A are input to the signal processing device 200 from the photoelectric conversion member shown in FIG. 8B. Then, the Blue interpolation circuit 166 of the signal processing device 200 performs Blue interpolation processing shown in C in FIG. 8 on the blue light signal Bin.
Next, the Blue sub-absorption correction circuit 170 of the signal processing device 200 performs the Blue absorption correction shown in D in FIG. 8 on the green light signal Gin using the blue interpolated blue light signal Bo. I do. In addition, in the Blue absorption correction process shown in D of FIG. 8, the absorption correction process may also be performed on the interpolated blue light signal Bo.
Next, the Red interpolation circuit 174 of the signal processing device 200 performs Red interpolation processing shown in E of FIG. 8 on the red light signal Rin.
Next, the color correction matrix circuit of the signal processing device 200 performs the color correction matrix processing shown in F in FIG. 8 on the blue light signal Bo, the red light signal Ro, and the green light signal Go. (color correction processing). The specific color correction processing in the signal processing device 200 is the same as the color correction processing in the signal processing device 100 according to the first embodiment, so the description thereof will be omitted.

Next, with reference to FIG. 9, the interpolation process shown in C of FIG. 8, the absorption correction process shown in D of FIG. 8, and the interpolation process shown in E of FIG. 8 will be conceptually explained.
As shown in FIG. 9A, the signal processing device 200 according to the second embodiment first performs interpolation on the blue light signal Bin. When red pixels and blue pixels are arranged in a checkerboard pattern in the first photoelectric conversion layer L1, there are red pixels and blue pixels that have no pixel value. Therefore, the signal processing device 200 first performs interpolation on the blue light signal Bin. Note that the signal processing device 200 may first perform interpolation on the red light signal Rin.

For example, the signal processing device 200 performs interpolation on the blue light signal Bin while referring to the gradient (intensity gradient information) of the pixel value of the green pixel indicated by the green light signal Gin. The reason why interpolation is performed by referring to the gradient of the pixel value of the green pixel is that, for example, all green pixels have pixel values, images of each color are correlated, and green has the highest contribution to brightness. It is. Furthermore, since the green light signal Gin includes the blue light component absorbed by the second photoelectric conversion layer L2, by performing interpolation with reference to the gradient of the pixel value of the green pixel, Interpolation can be performed with higher precision. The specific interpolation process in the signal processing device 200 is the same as the interpolation process in the signal processing device 100 according to the first embodiment, so a description thereof will be omitted.

When interpolation is performed on the blue light signal Bin, the signal processing device 200 calculates the green light signal based on the green light signal Gin and the interpolated blue light signal Bo, as shown in FIG. 9B. Absorption correction processing is performed on the optical signal Gin. The specific absorption correction process in the signal processing device 200 is the same as the absorption correction process in the signal processing device 100 according to the first embodiment, so the description thereof will be omitted. By performing absorption correction processing on the green light signal Gin, a green light signal with high color reproducibility is generated.

When the absorption correction process is performed on the green light signal Gin, the signal processing device 200 performs interpolation on the red light signal Rin, as shown in C in FIG.

For example, the signal processing device 200 performs calculations on the red light signal Rin while referring to the gradient (intensity gradient information) of the pixel value of the green pixel indicated by the green light signal Go after absorption correction processing. Perform interpolation. In the pixel value of the green pixel after the absorption correction process is performed, the influence of the blue light component absorbed by the second photoelectric conversion layer L2 is removed. Therefore, by performing interpolation with reference to the gradient of the pixel value of the green pixel after absorption correction processing has been performed, the gradient of the pixel value of the blue pixel can affect the negative effect on the interpolation of the red light signal Rin. , can be reduced. The specific interpolation process in the signal processing device 200 is the same as the interpolation process in the signal processing device 100 according to the first embodiment, so a description thereof will be omitted.

The signal processing device 200 and the signal processing method according to the second embodiment described above not only provide the same effects as the signal processing device 100 and the signal processing method according to the first embodiment, but also absorb Since the red light signal Rin is interpolated using the corrected green light signal Go, the red light signal Rin is The optical signal Rin can be interpolated. Thereby, the red light signal Rin can be interpolated with even higher accuracy.

Examples 1, 2 and comparative example 1
Next, Example 1, Example 2, and Comparative Example 1 of the present invention will be described.
A signal processing device according to Example 1 of the present invention is signal processing device 100 according to Embodiment 1. Further, the signal processing device according to the second embodiment of the present invention is the signal processing device 200 according to the second embodiment.

A signal processing method in the signal processing device according to Comparative Example 1 will be described with reference to FIG. 10.
First, a red light signal Rin, a green light signal Gin, and a blue light signal Bin generated by the photoelectric conversion member from the imaging unit shown in A in FIG. 10 are input to the signal processing device according to Comparative Example 1. Ru. Note that the imaging unit and photoelectric conversion member included in the imaging unit of Comparative Example 1 are the same as those of Example 1 and Example 2. That is, in the first photoelectric conversion layer L1 of the photoelectric conversion member, blue pixels and red pixels are arranged, for example, in a checkerboard pattern, and green pixels are arranged throughout all the pixels of the second photoelectric conversion layer L2. pixels are arranged.

Next, as shown in FIG. 10B, the signal processing device according to Comparative Example 1 performs blue absorption correction on the green light signal Gin using the blue light signal Bin input from the imaging unit. I do. Here, the blue light signal Bin used in the blue absorption correction process for the green light signal Gin is not interpolated. That is, in the signal processing device according to Comparative Example 1, Blue absorption correction is performed on the green light signal Gin in a checkered pattern.

Next, as shown in C of FIG. 10, the signal processing device according to Comparative Example 1 performs interpolation processing on the blue light signal Bin and the red light signal Rin input from the imaging unit.

Next, as shown in FIG. 10D, the signal processing device according to Comparative Example 1 generates a blue light absorption-corrected green light signal Go, an interpolated blue light signal Bo, and an interpolated blue light signal Bo. Color correction matrix processing (color correction processing) is performed on the red light signal Ro.

FIG. 11 shows a region with pixel values (R, G, B) = (0%, 25%, 100%) and a pixel value ( This shows an image that is output when a chart in which regions of R, G, B)=(0%, 0%, 0%) are alternately arranged is input. Note that in the chart, blue pixels and red pixels are arranged in a checkered pattern. The right side of FIG. 11 shows an image output from the signal processing device according to Example 1, and the left side of FIG. 11 shows an image output from the signal processing device according to Comparative Example 1. Note that, in the image shown in FIG. 11, color correction matrix processing (color correction processing) is not performed on the input image (the above chart). That is, the image shown on the right side of FIG. 11 is obtained by performing interpolation processing on the blue light signal Bin using the intensity gradient information of the green light signal Gin of the input image (the above chart). This is an image obtained by performing absorption correction on the green light signal Gin using the blue light signal Bo. In addition, the image shown on the left side of FIG. 11 shows the absorption-corrected green light after performing absorption correction on the green light signal Gin using the blue light signal Bin of the input image (the above chart). This is an image obtained by performing interpolation processing on the blue light signal Bin using the signal Go.
As shown in FIG. 11, it can be seen that Example 1 is superior to Comparative Example 1 in color reproducibility and color resolution.

FIG. 12 shows a region with pixel values (R, G, B) = (100%, 25%, 100%) and a pixel value ( This shows an image that is output when a chart in which regions of R, G, B)=(0%, 0%, 0%) are alternately arranged is input. Note that in the chart, blue pixels and red pixels are arranged in a checkered pattern. The right side of FIG. 12 shows an image output from the signal processing device according to Example 1, and the left side of FIG. 12 shows an image output from the signal processing device according to Comparative Example 1. Note that, in the image shown in FIG. 12, color correction matrix processing (color correction processing) is not performed on the input image (the above chart). That is, the image shown on the right side of FIG. 12 is obtained by performing interpolation processing on the blue light signal Bin and the red light signal Rin using the intensity gradient information of the green light signal Gin of the input image (the above chart). This image is obtained by performing absorption correction on the green light signal Gin using the interpolated blue light signal Bo. In addition, the image shown on the left side of FIG. 12 shows the absorption-corrected green light after performing absorption correction on the green light signal Gin using the blue light signal Bin of the input image (the above chart). This is an image obtained by performing interpolation processing on the blue light signal Bin and the red light signal Rin using the signal Go.
As shown in FIG. 12, it can be seen that Example 1 is superior to Comparative Example 1 in color reproducibility and color resolution.

FIG. 13 shows a region with pixel values (R, G, B) = (100%, 0%, 0%) and a pixel value ( This shows an image that is output when a chart in which regions of R, G, B)=(0%, 0%, 100%) are alternately arranged is input. Note that in the chart, blue pixels and red pixels are arranged in a checkered pattern. The right side of FIG. 13 shows an image output from the signal processing device according to the second embodiment, and the left side of FIG. 13 shows an image output from the signal processing device according to the first embodiment. Note that, in the image shown in FIG. 13, color correction matrix processing (color correction processing) is not performed on the input image (the above chart). That is, the image shown on the right side of FIG. 13 is obtained by performing interpolation processing on the blue light signal Bin using the intensity gradient information of the green light signal Gin of the input image (the above chart). Absorption correction is performed on the green light signal Gin using the blue light signal Bo, and then interpolation is performed on the red light signal Rin using the absorption-corrected green light signal Go. This is an image obtained by processing. In addition, the image shown on the left side of FIG. 13 is obtained by performing interpolation processing on the blue light signal Bin and the red light signal Rin using the intensity gradient information of the green light signal Gin of the input image (the above chart). This image is obtained by performing absorption correction on the green light signal Gin using the interpolated blue light signal Bo.
As shown in FIG. 13, when an image in which blue areas and red areas are arranged alternately as shown in the chart is input, the color reproduction in Example 2 is better than in Example 1. It can be seen that the image quality and color resolution are excellent.

Embodiment 3
Next, a third embodiment of the present invention will be described with reference to FIGS. 14 to 17. FIG. 14 is a diagram illustrating an example of a signal processing device 300 according to the third embodiment. FIG. 15 is a diagram illustrating an example of a signal processing method according to the third embodiment. FIG. 16 is a diagram conceptually showing the interpolation process shown in C of FIG. 15, the absorption correction process shown in D of FIG. 15, the interpolation process shown in E of FIG. 15, and the absorption correction process shown in F of FIG. 15. Further, FIG. 17 shows an example of the result of Red absorption correction in the signal processing device according to the third embodiment.

As shown in FIG. 14, the signal processing device 300 according to the third embodiment differs from the signal processing device 200 according to the second embodiment in that it includes a Red sub-absorption correction circuit 176 as a second absorption correction section. . Hereinafter, in the signal processing device 300 according to the third embodiment, the same components as those in the signal processing device 200 according to the second embodiment will be denoted by the same reference numerals, and the description thereof will be omitted.

The Red sub-absorption correction circuit 176 performs absorption correction processing on the green light signal Go based on the green light signal Go read out from the Green holding line memory 172 and the interpolated red light signal Ro. I do. Note that the red sub-absorption correction circuit 176 may perform absorption correction processing on the interpolated red light signal Ro.

The blue light signal Bo subjected to interpolation processing in the Blue interpolation circuit 166 is inputted to a color correction matrix circuit (not shown) via the Blue delay line memory 168, and the green light signal Bo subjected to absorption correction processing is output from the Red sub-absorption correction circuit 176. The light signal Go is input to a color correction matrix circuit (not shown), and the red light signal Ro interpolated by the Red interpolation circuit 174 is input to a color correction matrix circuit (not shown). Then, the color correction matrix circuit performs color correction processing on the blue light signal Bo, the red light signal Ro, and the green light signal Go.

As shown in FIG. 15, the signal processing method according to the third embodiment is different from the second embodiment in that the Red absorption correction shown in F in FIG. 15 is performed after the Red interpolation processing shown in E in FIG. Different signal processing method. Note that in the third embodiment, the processing up to the Red interpolation process is the same as the processing in the second embodiment. Therefore, in the signal processing method according to the third embodiment, descriptions of the same processes as the signal processing method according to the second embodiment will be omitted.

The signal processing method according to the third embodiment includes a blue interpolation process as a first interpolation step shown in C of FIG. 15, a blue absorption correction process as a first absorption correction step shown in D of FIG. It includes Red interpolation processing as a second interpolation step shown in E, Red absorption correction processing as a second absorption correction step shown in F in FIG. 15, and color correction matrix processing as a color correction step shown in G in FIG. . Note that the Blue interpolation process, Blue absorption correction process, and Red interpolation process are the same as the Blue interpolation process, Blue absorption correction process, and Red interpolation process shown in FIG. 8, and therefore their explanations will be omitted.
Next, the Red sub-absorption correction circuit 176 of the signal processing device 300 uses the red light signal Ro subjected to the Red interpolation process to correct the green light signal Go subjected to the Blue absorption correction, as shown in FIG. Red absorption correction shown in F is performed. Here, the Red absorption correction, like the Blue absorption correction, is performed by subtracting the value obtained by multiplying the value of the red signal Rin by α'/(1-α') from the green light signal Go. It will be done. However, unlike the Blue absorption correction, the correction value α' is determined by calculating the values of YSNR10 and color difference ΔE ^* ab, rather than Gbottom' in equation (13) or Gabs' in equation (14). Specifically, the correction value α' is within a range where the value of the color difference ΔE ^* ab does not deteriorate and is a value at which the YSNR10 is the lowest. Here, YSNR10 means the illuminance (unit: lux) at which the ratio of the green signal after color correction matrix processing to the noise generated when measuring the signal in the image sensor (photoelectric conversion member) is 10. do. Furthermore, the smaller the YSNR10 value, the better the image characteristics at low illuminance.
Note that in the Red absorption correction process shown in F in FIG. 15, the absorption correction process may also be performed on the interpolated red light signal Ro.
Next, the color correction matrix circuit of the signal processing device 300 processes the blue light signal Bo, the red light signal Ro, and the green light signal Go subjected to the Red absorption correction as shown in FIG. Color correction matrix processing (color correction processing) shown in G is performed. Note that the specific color correction processing in the signal processing device 300 according to the third embodiment differs from the color correction processing in the signal processing devices 100 and 200 according to the first and second embodiments, in that the value of the color difference ΔE ^* ab is This is carried out so that YSNR10 is the lowest within a range that does not deteriorate.

Next, the absorption correction process shown in F in FIG. 15 will be conceptually explained with reference to FIG. 16. Note that the interpolation process shown in C in FIG. 15, the absorption correction process shown in D in FIG. 15, and the interpolation process shown in E in FIG. 15 are the interpolation process shown in C in FIG. 8, the absorption correction process shown in D in FIG. Since this is the same as the interpolation process shown in E of FIG. 8, its explanation will be omitted.

When interpolation is performed on the red light signal Rin, the signal processing device 300 generates the green light signal Go subjected to the Blue absorption correction and the interpolated red light signal, as shown in FIG. 16D. Based on Ro, absorption correction processing is performed on the green light signal Go. By performing absorption correction processing on the green light signal Go, a green light signal with high color reproducibility is generated.

FIG. 17 shows the result of Red absorption correction performed in the signal processing device 300 according to the third embodiment. The left side of FIG. 17 shows a red light signal Ro, a green light signal Go, and a blue light signal Bo, which have been subjected to Blue interpolation processing, Blue absorption correction processing, and Red interpolation processing in the signal processing device 300. shows the intensity distribution of Further, the right side of FIG. 17 shows the intensity distribution of the red light signal Ro, the green signal Go that has undergone the Red absorption correction process, and the blue light signal Bo in the signal processing device 300. As shown in FIG. 17, by further subjecting the green signal Go that has undergone the Blue absorption correction processing to the Red absorption correction processing, the color purity of the red light signal Ro is improved, and the dark areas of the green light signal Go are The reproducibility (YSNR10 value) has improved. More specifically, in the left graph of FIG. 17, the result of subtracting the red light signal Ro from the value of the green light signal Go is shown in the right graph of FIG. As shown in the graph on the right side of FIG. 17, by subtracting the red light signal Ro from the value of the green light signal Go, the intensity of the green light signal Go becomes less than zero near the wavelength of 650 nm. You can see that it is happening. That is, in the signal processing method according to the third embodiment, when the red light signal Ro is subtracted from the value of the green light signal Go, the intensity of the green light signal Go becomes smaller on the wavelength side longer than 650 nm. is less than zero, that is, the sensitivity of the green light signal Go becomes negative. In normal absorption correction, the green light signal Go is not corrected to become negative, but in the signal processing method according to the third embodiment, the green light signal Go is By performing absorption correction so that the Go sensitivity becomes negative, a green light signal with high color reproducibility can be generated.

表１に示すように、実施の形態３に係る信号処理装置及び信号処理方法では、色差ΔＥ^＊ａｂの値が悪化しない範囲内において、ＴＳＮＲ１０を十分に低減することができている。すなわち、実施の形態３に係る信号処理装置及び信号処理方法では、赤色の光の信号Ｒｏの色純度を向上するとともに、緑色の光の信号Ｇｏの暗部での再現性を向上することができる。 Table 1 shows the implementation process when absorption correction is not performed for the red light signal Rin, green light signal Gin, and blue light signal Bin measured by two image sensors (photoelectric conversion members). When the signal processing device 100 according to Embodiment 1 performs Blue interpolation processing and Blue absorption correction processing, when the signal processing device 100 according to Embodiment 1 performs Red interpolation processing and Red absorption correction processing, Embodiment 3 The values of TSNR10 and color difference ΔE ^* ab are shown when Blue interpolation processing, Blue absorption correction processing, Red interpolation processing, and Red absorption correction processing are performed by the signal processing device according to .

As shown in Table 1, in the signal processing device and signal processing method according to the third embodiment, the TSNR10 can be sufficiently reduced within a range in which the value of the color difference ΔE ^* ab does not deteriorate. That is, the signal processing device and signal processing method according to the third embodiment can improve the color purity of the red light signal Ro and improve the reproducibility of the green light signal Go in dark areas.

The signal processing device and signal processing method according to the third embodiment described above not only provide the same effects as the signal processing device 200 and the signal processing method according to the second embodiment, but also provide interpolation processing. Not only is absorption correction processing performed on the green light signal Gin using the blue light signal Bo, but also absorption correction processing is further performed using the interpolated red light signal Ro. This makes it possible to generate a green light signal with even higher color reproducibility.

It should be noted that the signal processing device according to the embodiment of the present invention can be used to connect multiple computers, such as a “PC (Personal Computer), a server, etc.” or a “tablet-type device,” for example. The present invention can be applied to any device capable of processing image signals obtained by imaging with an image sensor having a photoelectric conversion layer. Further, the signal processing device according to the embodiment of the present invention can also be applied to, for example, an IC (Integrated Circuit) that can be incorporated into the above devices.

Further, as described above, the signal processing device according to the embodiment of the present invention includes an imaging unit configured with an image sensor having a first photoelectric conversion layer and a second photoelectric conversion layer (an example of a plurality of photoelectric conversion layers). It may also function as an imaging device. In this case, the imaging device according to the embodiment of the present invention may be, for example, a "Digital Still Camera", a "Digital Video Camera", or a "Smartphone" or a mobile phone. It can be applied to any device with this function.

Note that the present invention is not limited to the above embodiments, and can be modified as appropriate without departing from the spirit.

100, 200 Signal processing device 152 Blue/Red holding line memory 154 Green holding line memory 156 Write counter 158 Read counter 160 Blue/Red interpolation circuit (interpolation section)
160A Green inter-pixel correlation information extraction unit 160B Interpolation ratio calculation unit 164 Sub-absorption correction circuit (absorption correction unit)
166 Blue interpolation circuit (first interpolation section)
168 Blue delay line memory 170 Blue sub-absorption correction circuit (first absorption correction section)
172 Green delay line memory 174 Red interpolation circuit (second interpolation section)
176 Red sub-absorption correction circuit (second absorption correction section)
L1 First photoelectric conversion layer L2 Second photoelectric conversion layer

Claims (18)

a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light;
a second photoelectric conversion layer that is provided on a side where incident light is incident than the first photoelectric conversion layer and that photoelectrically converts third light having a wavelength different from the first light and the second light; ,
A signal processing device that processes a signal generated by a photoelectric conversion member comprising:
At least one of the first light signal and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer is converted into the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer. an interpolation unit that performs interpolation using a light signal;
an absorption correction unit that performs absorption correction of the third light signal that is not interpolated using at least one of the first light signal and the second light signal interpolated by the interpolation unit;
a color correction unit that performs color correction processing by calculating a color correction matrix on the interpolated first light signal, the interpolated second light signal, and the absorption-corrected third light signal;
A signal processing device comprising: The interpolation unit uses intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer to calculate the intensity of the first signal obtained by photoelectric conversion by the first photoelectric conversion layer. The signal processing device according to claim 1, wherein the signal processing device interpolates at least one of the optical signal of the second optical signal and the second optical signal. The absorption correction section uses a correction value α determined in consideration of at least one of the first light component and the second light component absorbed by the second photoelectric conversion layer. Claim: 1. Absorption correction of the third light signal is performed by subtracting a value obtained by multiplying the first light signal value by α/(1-α) from the third light signal. 3. The signal processing device according to 1 or 2. a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light;
a second photoelectric conversion layer that is provided on a side where incident light is incident than the first photoelectric conversion layer and that photoelectrically converts third light having a wavelength different from the first light and the second light; ,
A signal processing device that processes a signal generated by a photoelectric conversion member comprising:
A first method of interpolating the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer using the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer. an interpolation section;
a first absorption correction unit that performs absorption correction of the non-interpolated third light signal using the first light signal interpolated by the first interpolation unit;
a second interpolation unit that interpolates the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer using the third light signal absorption-corrected by the first absorption correction unit; and,
a color correction unit that performs color correction processing by calculating a color correction matrix on the interpolated first light signal, the interpolated second light signal, and the absorption-corrected third light signal;
A signal processing device comprising: The first interpolation section uses intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer, and the third light signal is obtained by photoelectric conversion by the first photoelectric conversion layer. The signal processing device according to claim 4, which interpolates the first light signal. The second interpolation unit uses intensity gradient information of the third light signal absorption-corrected by the first absorption correction unit to calculate the intensity of the third light signal obtained by photoelectric conversion by the first photoelectric conversion layer. 6. The signal processing device according to claim 4, wherein the signal processing device interpolates the light signal of the second signal. The first absorption correction unit adjusts the signal value of the first light using a correction value α determined in consideration of the component of the first light absorbed by the second photoelectric conversion layer. 7. Absorption correction of the third light signal is performed by subtracting a value obtained by multiplying α/(1−α) from the third light signal. The signal processing device described in . a second absorption correction that uses the second light signal interpolated by the second interpolation section to perform absorption correction of the third light signal that has been absorption-corrected by the first absorption correction section; The signal processing device according to any one of claims 4 to 7, further comprising a section. The first photoelectric conversion layer photoelectrically converts blue light as the first light and red light as the second light,
The second photoelectric conversion layer photoelectrically converts green light as the third light.
The signal processing device according to any one of claims 1 to 8. a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light;
a second photoelectric conversion layer that is provided on a side where incident light is incident than the first photoelectric conversion layer and that photoelectrically converts third light having a wavelength different from the first light and the second light; ,
A signal processing method for processing a signal generated by a photoelectric conversion member comprising:
At least one of the first light signal and the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer is converted into the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer. an interpolation step of interpolating using a light signal;
an absorption correction step of performing absorption correction of the third light signal that has not been interpolated, using at least one of the first light signal and the second light signal interpolated in the interpolation step;
a color correction step of performing color correction processing by calculating a color correction matrix on the interpolated first light signal, the interpolated second light signal, and the absorption-corrected third light signal;
A signal processing method comprising: In the interpolation step, intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to calculate the intensity of the first signal obtained by photoelectric conversion by the first photoelectric conversion layer. 11. The signal processing method according to claim 10, wherein at least one of the optical signal and the second optical signal is interpolated. In the absorption correction step, using a correction value α determined in consideration of at least one of the first light component and the second light component absorbed by the second photoelectric conversion layer, Claim: 1. Absorption correction of the third light signal is performed by subtracting a value obtained by multiplying the first light signal value by α/(1-α) from the third light signal. 10. The signal processing method according to 11. a first photoelectric conversion layer that photoelectrically converts first light and second light having a different wavelength from the first light;
a second photoelectric conversion layer that is provided on a side where incident light is incident than the first photoelectric conversion layer and that photoelectrically converts third light having a wavelength different from the first light and the second light; ,
A signal processing method for processing a signal generated by a photoelectric conversion member comprising:
A first method of interpolating the first light signal obtained by photoelectric conversion by the first photoelectric conversion layer using the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer. an interpolation step;
a first absorption correction step of performing absorption correction of the third light signal that has not been interpolated using the first light signal interpolated in the first interpolation step;
a second interpolation step of interpolating the second light signal obtained by photoelectric conversion by the first photoelectric conversion layer using the third light signal absorption-corrected in the first absorption correction step; and,
a color correction step of performing color correction processing by calculating a color correction matrix on the interpolated first light signal, the interpolated second light signal, and the absorption-corrected third light signal;
A signal processing method comprising: In the first interpolation step, intensity gradient information of the third light signal obtained by photoelectric conversion by the second photoelectric conversion layer is used to obtain the signal obtained by photoelectric conversion by the first photoelectric conversion layer. The signal processing method according to claim 13, wherein the first light signal is interpolated. In the second interpolation step, intensity gradient information of the third light signal that has been absorption-corrected in the first absorption correction step is used to calculate the intensity of the third light signal obtained by photoelectric conversion by the first photoelectric conversion layer. The signal processing method according to claim 13 or 14, wherein the signal of the second light is interpolated. In the first absorption correction step, the signal value of the first light is adjusted using a correction value α determined in consideration of the component of the first light absorbed by the second photoelectric conversion layer. Any one of claims 13 to 15, wherein absorption correction of the third light signal is performed by subtracting a value obtained by multiplying α/(1−α) from the third light signal. The signal processing method described in. a second absorption correction that performs absorption correction of the third light signal subjected to absorption correction in the first absorption correction step using the second light signal interpolated in the second interpolation step; The signal processing method according to any one of claims 13 to 16, further comprising a step. The first photoelectric conversion layer photoelectrically converts blue light as the first light and red light as the second light,
The second photoelectric conversion layer photoelectrically converts green light as the third light.
The signal processing method according to any one of claims 10 to 16.

Images