WO2023100640A1 - Semiconductor device, signal processing method, and program - Google Patents

Abstract

The present technology relates to a semiconductor device, a signal processing method, and a program for making it possible to obtain desired spectral characteristics. The semiconductor device comprises a signal processing unit that performs, on an image signal obtained by imaging performed by a pixel array unit having a plurality of unit pixels, signal processing to correct spectral characteristics on the basis of the temperature of the pixel array unit. The present technology may be applied to solid-state imaging devices.

Description

Semiconductor device, signal processing method, and program

The present technology relates to a semiconductor device, a signal processing method, and a program, and more particularly to a semiconductor device, a signal processing method, and a program that enable desired spectral characteristics to be obtained.

Conventionally, for a color image captured by an imaging device, color reproduction characteristics are required according to the intended use of the image.

Therefore, by providing a spectral sensitivity correction filter in the front stage of the imaging lens in the imaging device, the spectral characteristics are corrected in consideration of the spectral characteristics of the color filter and the image sensor, and the desired color reproduction characteristics are obtained. has been proposed (see, for example, Patent Document 1).

JP 2015-100088 A

However, with the above-described techniques, it was sometimes impossible to obtain the desired spectral characteristics for obtaining the desired color reproduction characteristics.

Specifically, the spectral characteristics of the image sensor fluctuate depending on the temperature. Therefore, for example, when the temperature of the image pickup device rises, the spectral sensitivity correction filter designed in advance cannot cope with the fluctuation of the spectral characteristics of the image pickup device, making it impossible to obtain the desired spectral characteristics. end up

This technology has been developed in view of such circumstances, and enables the desired spectral characteristics to be obtained.

A semiconductor device according to one aspect of the present technology is a signal that performs signal processing for correcting spectral characteristics based on the temperature of the pixel array unit for an image signal obtained by imaging a pixel array unit having a plurality of unit pixels. It has a processing unit.

A signal processing method or program according to one aspect of the present technology is signal processing for correcting the spectral characteristics of an image signal obtained by imaging a pixel array section having a plurality of unit pixels based on the temperature of the pixel array section. including the step of

In one aspect of the present technology, signal processing for correcting spectral characteristics based on the temperature of the pixel array section is performed on an image signal obtained by imaging with a pixel array section having a plurality of unit pixels.

It is a figure which shows the structural example of a solid-state imaging device. It is a figure which shows the other structural example of a solid-state imaging device. 4 is a flowchart for explaining imaging processing; It is a figure explaining the specific example of signal processing. It is a figure which shows the other structural example of a solid-state imaging device. It is a figure which shows the structural example of a unit pixel. It is a figure explaining the change of the spectral characteristic with respect to the change of temperature. It is a figure which shows the other structural example of a solid-state imaging device. It is a figure which shows the structural example of an imaging device. It is a figure explaining the usage example of a solid-state imaging device. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; FIG. FIG. 4 is an explanatory diagram showing an example of installation positions of an outside information detection unit and an imaging unit;

Embodiments to which the present technology is applied will be described below with reference to the drawings.

<First embodiment>
<Configuration example of solid-state imaging device>
The present technology provides desired spectral characteristics by providing a thermometer in an imaging device and correcting the spectral characteristics of an image according to the temperature of the imaging device. As a result, it is possible to obtain the desired color reproduction characteristics according to the intended use of the image.

FIG. 1 is a diagram showing a configuration example of an embodiment of a solid-state imaging device to which the present technology is applied.

The solid-state imaging device 11 shown in FIG. 1 is a solid-state imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

The solid-state imaging device 11 includes a pixel array section 21, a vertical driving section 22, a column processing section 23, a data storage section 24, a horizontal driving section 25, a system control section 26, a signal processing section 27, a recording section 28, and a thermometer 29-1. , and a thermometer 29-2. Incidentally, hereinafter, the thermometers 29-1 and 29-2 are also simply referred to as the thermometers 29 when there is no particular need to distinguish between them.

In this example, for example, the pixel array section 21, the vertical drive section 22 to system control section 26 as the peripheral circuit section, and the thermometer 29 are formed on the same semiconductor substrate (chip). Also, the signal processing section 27 and the recording section 28 are provided on a semiconductor substrate or the like different from the semiconductor substrate on which the pixel array section 21 is formed, for example.

The pixel array section 21 includes a plurality of unit pixels 41 (hereinafter sometimes simply referred to as pixels 41) each having a photoelectric conversion section that generates and accumulates an electric charge corresponding to the amount of received light, arranged in the row direction and the column direction. , that is, they are arranged two-dimensionally in a matrix.

Here, the row direction is the arrangement direction (horizontal direction) of the pixels 41 in the pixel row, that is, the horizontal direction in the drawing, and the column direction is the arrangement direction (vertical direction) of the pixel column, that is, Vertically.

In the pixel array section 21, pixel drive lines 42 are wired along the row direction for each pixel row, and vertical signal lines 43 are wired along the column direction for each pixel column with respect to the matrix-like pixel arrangement. . The pixel drive lines 42 are signal lines for supplying drive signals (control signals) for driving the pixels 41 , such as driving when reading signals from the pixels 41 . One end of the pixel drive line 42 is connected to an output terminal corresponding to each row of the vertical drive section 22 .

Although one pixel drive line 42 is drawn for one pixel row here for the sake of clarity, a plurality of pixel drive lines 42 are actually wired for one pixel row. there is

The vertical driving section 22 is composed of, for example, a shift register and an address decoder, and drives each pixel 41 of the pixel array section 21 simultaneously or in units of rows.

For example, the vertical drive section 22 is configured to have two scanning systems, a readout scanning system and a discharge scanning system.

The readout scanning system sequentially selectively scans the unit pixels 41 of the pixel array section 21 row by row in order to read signals from the unit pixels 41 . A signal read from the unit pixel 41 is an analog signal.

The sweep-scanning system performs sweep-scanning at a predetermined timing on the read-out rows to be read-scanned by the read-out scanning system. The sweep scan by the sweep scan system sweeps out unnecessary electric charges from the photoelectric converters of the unit pixels 41 in the readout row, thereby resetting the photoelectric converters.

A signal output from each unit pixel 41 in a pixel row selectively scanned by the vertical drive unit 22 is input to the column processing unit 23 via the vertical signal line 43 for each pixel column.

The column processing unit 23 performs predetermined signal processing on signals supplied from the pixels 41 of the selected row through the vertical signal lines 43 for each pixel column of the pixel array unit 21, and processes the pixels after the signal processing. The signal is supplied to the data storage unit 24 to hold it.

For example, the column processing unit 23 performs noise removal processing such as CDS (Correlated Double Sampling) processing (correlated double sampling) and AD (Analog to Digital) conversion processing as signal processing. For example, the CDS processing removes fixed pattern noise unique to the pixels 41 such as reset noise and variations in threshold values of amplification transistors in the pixels 41 .

The data storage unit 24 temporarily holds the pixel signal obtained for each pixel 41 supplied from the column processing unit 23, that is, the image signal composed of the pixel signal of each pixel 41, and stores the held image signal. Output (supply) to the signal processing unit 27 .

The horizontal drive section 25 is composed of a shift register, an address decoder, etc., and selects unit circuits corresponding to the pixel columns of the data storage section 24 in order. By selective scanning by the horizontal driving section 25 , the pixel signals held for each unit circuit in the data storage section 24 are sequentially output to the signal processing section 27 .

The system control unit 26 includes a timing generator that generates various timing signals, and controls driving of the vertical driving unit 22, the column processing unit 23, the data storage unit 24, the horizontal driving unit 25, etc. based on the generated timing signals. I do.

The pixel array unit 21 is provided with, for example, an R (red) color filter, for example, a pixel 41 that outputs an R component signal, and a G (green) color filter that outputs a G component signal. A pixel 41 and a B (blue) color filter are provided, and the pixel 41 that outputs a B component signal is provided.

Therefore, for example, the solid-state imaging device 11 can obtain an RGB color image composed of R component pixel signals, G component pixel signals, and B component pixel signals. The colors of the color filters provided in the pixels 41 are not limited to R, G, and B, and may be any other colors.

A thermometer 29-1 and a thermometer 29-2 are provided in the vicinity of the pixel array section 21.

In this example, a thermometer 29-1 and a thermometer 29-2 are provided (arranged) at positions adjacent to each other in the vicinity of the pixel array section . By arranging the thermometers 29 in this way, even if one thermometer 29 malfunctions, the temperature of the pixel array section 21 can be measured by the other thermometer 29 .

For example, each thermometer 29 (temperature sensor) is embedded inside the semiconductor substrate on which the pixel array section 21 is formed, but each thermometer 29 is arranged on the surface of the semiconductor substrate on which the pixel array section 21 is formed. You may do so.

The thermometer 29 measures the temperature of the semiconductor substrate on which the pixel array section 21 is formed, and supplies the measurement result to the signal processing section 27 via a signal line or the like (not shown).

Since the thermometer 29 is arranged adjacent to the pixel array section 21 , the temperature of the semiconductor substrate measured by the thermometer 29 is the temperature of the photoelectric conversion section within the pixel 41 formed in the pixel array section 21 . be able to.

Although an example in which two thermometers 29 are provided is described here, the number of thermometers 29 may be one, or three or more.

Also, the position where the thermometer 29 is arranged may be any position as long as it is in the vicinity of the pixel array section 21 .

For example, the thermometers 29-1 and 29-2 may be arranged at mutually different positions in the vicinity of the pixel array section 21. Specifically, for example, a thermometer 29-1 is provided at a position adjacent to the upper right side of the pixel array section 21 in the drawing, and a thermometer 29-2 is provided at a position adjacent to the lower left side of the pixel array section 21 in the drawing. may be provided.

Further, for example, the thermometer 29 is arranged at a position adjacent to the pixel array section 21, near the system control section 26 having a PLL (Phase Locked Loop) circuit, etc., or at a position near the column processing section 23 having an AD conversion circuit. may That is, for example, the thermometer 29 may be arranged at a position near the pixel array section 21 and near a circuit provided around the pixel array section 21 where temperature changes are likely to occur.

The signal processing unit 27 has at least an arithmetic processing function, and performs various signal processing on the image signal supplied from the data storage unit 24, that is, the color image captured by the pixel array unit 21 (solid-state imaging device 11). , and outputs the resulting image signal to the subsequent stage.

For example, the signal processing unit 27 includes processing for correcting the spectral characteristics of the image signal (image) based on the temperature supplied from the thermometer 29 and a lookup table prerecorded in the recording unit 28. Perform signal processing. At this time, processing for correcting the spectral characteristics is performed for each color component of the image, for example. Further, for example, the signal processing unit 27 reads and executes a program recorded in the recording unit 28 to perform various signal processing on the image signal.

The lookup table used during signal processing is prepared in advance for each of a plurality of possible temperatures of the pixel array section 21 and is used to obtain appropriate coefficients corresponding to the temperatures of the pixel array section 21 during signal processing. is. In addition, in the lookup table, each temperature may be stored in association with an appropriate coefficient obtained in advance for each temperature.

The signal processing unit 27 performs signal processing using coefficients obtained by referring to the lookup table to correct the spectral characteristics of the image signal so that an image signal corresponding to the desired spectral characteristics is obtained.

By referring to such a lookup table, gain correction for each of the R, G, and B color components linked to the temperature of the pixel 41, that is, switching of linear matrix coefficients, etc., depending on the temperature can be performed. will be As a result, correction is realized to return the spectral characteristics of the image signal to ideal (desired) characteristics. Here, the ideal spectral characteristics are, for example, spectral characteristics obtained when the pixel array section 21 is at a predetermined temperature such as room temperature.

<Another configuration example of the solid-state imaging device>
1, the pixel array section 21 and the signal processing section 27 are formed on different semiconductor substrates (chips), but the pixel array section 21 and the signal processing section 27 are formed on the same semiconductor substrate. You may do so.

In such a case, the solid-state imaging device 11 is configured, for example, as shown in FIG. In FIG. 2, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

2, as in the example of FIG. 1, the solid-state imaging device 11 includes a pixel array section 21, a vertical driving section 22, a column processing section 23, a data storage section 24, a horizontal driving section 25, a system It has a control section 26, a signal processing section 27, a recording section 28, a thermometer 29-1, and a thermometer 29-2.

However, in the solid-state imaging device 11 shown in FIG. 2, a signal processing section 27 is provided between the data storage section 24 and the horizontal driving section 25 . A timing signal is supplied from the system control unit 26 to the signal processing unit 27 .

The signal processing unit 27 performs various signal processing on the image signal supplied from the data storage unit 24 according to the instruction from the horizontal driving unit 25, and outputs the resulting image signal to the subsequent stage.

The solid-state imaging device 11 may have either the configuration shown in FIG. 1 or the configuration shown in FIG. 2, but the solid-state imaging device 11 has the configuration shown in FIG. Continue to explain.

<Description of imaging processing>
Next, operation of the solid-state imaging device 11 will be described. That is, the imaging process performed by the solid-state imaging device 11 will be described below with reference to the flowchart of FIG.

In step S11, the pixel array section 21 captures an image.

That is, each pixel 41 provided in the pixel array section 21 receives incident light from the outside (subject), photoelectrically converts the light, and accumulates the resulting charge. In addition, each pixel 41 supplies (outputs) a signal corresponding to the accumulated charge to the column processing section 23 via the vertical signal line 43 according to the driving signal supplied from the vertical driving section 22 .

Also, each thermometer 29 measures its own ambient temperature as the temperature of the pixel array section 21 (semiconductor substrate) and outputs the measurement result to the signal processing section 27 .

In step S12, the column processing unit 23 performs noise removal processing and AD conversion processing on the signal supplied from each pixel 41 of the pixel array unit 21, and stores the resulting pixel signal for each pixel 41 in the data storage unit. 24 to hold it.

In addition, the data storage unit 24 sequentially outputs the held pixel signals to the signal processing unit 27 according to the selective scanning by the horizontal driving unit 25, thereby providing the signal processing unit 27 with the pixel array unit 21 provides an image signal of the image picked up.

In step S<b>13 , the signal processing unit 27 acquires the current temperature of the pixel array unit 21 from the thermometer 29 .

For example, the signal processing unit 27 uses the average value and the maximum value of the temperatures simultaneously obtained from the plurality of thermometers 29 as the temperature of the pixel array unit 21 at the present point in subsequent processing.

In step S14, the signal processing unit 27 stores the image supplied from the data storage unit 24 based on the current temperature of the pixel array unit 21 obtained in step S13 and the lookup table recorded in the recording unit 28. Signal processing including processing for correcting spectral characteristics is performed on the signal. As a result, an image signal with desired spectral characteristics is obtained.

In step S15, the signal processing unit 27 outputs the image signal obtained by the process of step S14 to the subsequent stage, and the imaging process ends.

As described above, the solid-state imaging device 11 measures the temperature of the pixel array section 21 and performs signal processing including processing for correcting the spectral characteristics of the image signal based on the measurement result. Thus, an image signal corresponding to desired spectral characteristics can be obtained, and as a result, desired color reproduction characteristics can be obtained regardless of temperature fluctuations in the pixel array section 21 . In particular, it is possible to suppress the deterioration of the color reproduction characteristics due to the temperature rise of the pixel array section 21 .

<Specific example of signal processing including processing for correcting spectral characteristics>
Next, a specific example of signal processing including processing for correcting spectral characteristics, which is performed in step S14 of the imaging processing described with reference to FIG. 3, will be described.

For example, the signal processing unit 27 performs black level correction on the image signal, which is raw data supplied from the data storage unit 24, as shown in FIG.

Next, the signal processing unit 27 performs noise removal processing and shading correction on the image signal after black level correction.

At this time, the signal processing unit 27 performs shading correction according to the temperature of the pixel array unit 21 at the present time, for example, from a lookup table of the recording unit 28. At the same time, the coefficient (gain) for correcting the spectral characteristics is read out. . For example, the coefficients used for shading correction are different for each area of the image based on the image signal, such as the center and edge areas. The signal processing unit 27 performs shading correction on the image signal based on the read coefficients.

The correction coefficient (gain) for each temperature for correcting the spectral characteristics is stored in the lookup table, and the coefficient (PXSHD coefficient) used for shading correction is corrected by the correction coefficient according to the temperature. You may do so. In such a case, the read correction factor is multiplied by the PXSHD factor to correct the PXSHD factor. Shading correction is then performed using the corrected PXSHD coefficients.

Subsequently, the signal processing unit 27 performs HDR (High Dynamic Range) synthesis processing on the image signal.

In the HDR synthesis process, for example, an image captured with a relatively short exposure time in the pixel 41 (hereinafter also referred to as a short image) and an image captured in the pixel 41 with a longer exposure time than the short image (hereinafter referred to as a long image) (also referred to as stored image) are synthesized.

For example, a short-term image and a long-term image are images captured at different timings by bracketing or the like. The signal processing unit 27 performs the above-described black level correction, noise removal processing, and shading correction on each of the image signal of the short-term image and the image signal of the long-term image.

In addition, as will be described later in Modified Example 2 of the first embodiment, the unit pixel 41 has a sub-pixel structure having two pixels (sub-pixels) with different sensitivities, and the sub-pixels have approximately the same sensitivity. A short-term image and a long-term image may be captured at the timing.

The signal processing unit 27 reads, for example, from the lookup table of the recording unit 28, a combined gain corresponding to the current temperature of the pixel array unit 21. Then, the signal processing unit 27 multiplies the short-term image by the read synthetic gain, and adds (syntheses) the short-term image multiplied by the synthetic gain and the long-term image to achieve HDR with a wider dynamic range. Obtain a composite image. Through the HDR synthesis processing using such synthesis gains for each temperature, the spectral characteristics are corrected at the same time as the HDR synthesis image is generated.

Note that even during HDR synthesis processing, the correction coefficient (gain) for each temperature for correcting the spectral characteristics is read from the lookup table, and the read correction coefficient is multiplied by the predetermined synthesis gain. You may do so. In such a case, the composite gain multiplied by the correction coefficient, that is, the composite gain corrected by the correction coefficient is used to perform the HDR composite processing.

Furthermore, the signal processing unit 27 performs WB (White Balance) adjustment processing (white balance adjustment processing) on the image signal of the HDR composite image.

At this time, the signal processing unit 27 reads the correction gain for the WB adjustment process according to the current temperature of the pixel array unit 21 from, for example, the lookup table of the recording unit 28 . Then, the signal processing unit 27 corrects the WB gain by multiplying the read correction gain by the WB gain for the WB adjustment process obtained based on the image signal or the like, and calculates the corrected WB gain for each pixel of the image signal. Adjust the WB of the HDR composite image by multiplying the . As a result, the spectral characteristics are corrected simultaneously with the WB adjustment processing. A different WB gain is used for each color (color component) of each pixel forming the HDR composite image.

The signal processing unit 27 performs the processing described above as the signal processing in step S14 of FIG. 3, and outputs the resulting image signal to the subsequent stage.

In the above, an example in which the spectral characteristics are corrected regardless of the temperature of the pixel array section 21 has been described. You may choose not to do this.

Specifically, for example, when the temperature of the pixel array unit 21 is equal to or higher than a predetermined threshold th1 and is equal to or lower than a predetermined threshold th2, the signal processing unit 27 does not use the lookup table of the recording unit 28, 3, the signal processing of step S14 is performed. In this case, spectral characteristics are not corrected in shading correction, HDR synthesis processing, and WB adjustment processing performed as signal processing.

On the other hand, when the temperature of the pixel array section 21 is less than the threshold th1 or greater than the threshold th2, that is, when the temperature is outside the predetermined range, the signal processing section 27 performs a lookup of the recording section 28. Signal processing in step S14 of FIG. 3 is performed using the table. In this case, the spectral characteristics are corrected during signal processing, as described with reference to FIG. 4, for example.

By the way, in the signal processing unit 27, processing for correcting the spectral characteristics is performed on the image signal. By using a matrix, a method of interlocking coefficients with temperature, a method of separating for each wavelength band and multiplying by a lookup table gain, and the like can be considered.

For example, as examples of color filters provided in the pixels 41, in addition to the R (red), G (green), and B (blue) described above, C (colorless), Y (yellow), Mg (magenta), Cy (cyan), ), and various other color filters are conceivable.

In this case, as the pixel array in the pixel array section 21, not only an RGGB array called a so-called Bayer array, an RCCB in which G (green) of RGGB is replaced with C (colorless), but also four or more colors (color filters) are used. The pixel array used can be considered.

Therefore, for example, let the number of types of color filters provided in the pixel array unit 21 be N, and use an N×N (N rows by N columns) matrix whose elements are coefficients for each combination of the N types of colors, The spectral characteristics of the image signal may be corrected. This N×N matrix is a linear matrix, and the elements of the N×N matrix are linear matrix coefficients for correcting spectral characteristics. For example, the recording unit 28 records an N×N matrix for each temperature.

In such a case, the spectral characteristics of the image signal are corrected by calculating the product of the value of each color component in the image signal, that is, the matrix whose elements are the values of the pixel signals of each color, and the above N×N matrix. Processing is realized. For example, in addition to the processing described with reference to FIG. 4, such matrix computation using an N×N matrix may be performed as signal processing for correcting spectral characteristics.

In addition, for example, when the pixel array unit 21 is a multispectral sensor or the like, it is possible to obtain signals for more wavelength bands.

In such a case, the image signal can be separated for each wavelength band to correct the spectral characteristics. That is, a process of correcting the spectral characteristics is performed based on the signal for each wavelength band obtained by the pixel array unit 21 and the lookup table for each wavelength band. At this time, a gain corresponding to the temperature of the pixel array unit 21 is read from, for example, a lookup table for each wavelength band, and multiplied by the signal for each wavelength band.

It should be noted that the recording unit 28 that records the lookup table can be composed of, for example, OTP (One Time Programmable ROM (Read Only Memory)).

In such a case, for example, when the solid-state imaging device 11 is manufactured, a lookup table for each temperature for each signal processing such as shading correction and HDR synthesis processing prepared for each solid-state imaging device 11 by measurement or the like is stored in the recording unit 28. written (recorded) to the OTP as

After that, the lookup table recorded in the OTP is used as a fixed value for each temperature when executing each signal processing.

In addition, for example, when a lookup table written as initial values in the recording unit 28 is read out to a register or the like and used, the lookup table arbitrarily set (changed) by the user after the solid-state imaging device 11 is started. may be held in a register.

In this case, the lookup table after modification by the user, which is held in the register, is used to perform the signal processing in step S14 of FIG. By allowing the user to arbitrarily set the lookup table in this way, it becomes possible to correct the spectral characteristics more appropriately.

<Modification 1 of the first embodiment>
<Configuration example of solid-state imaging device>
The signal processing unit 27 and the recording unit 28 described above may be provided after the imaging device (solid-state imaging device).

In such a case, the solid-state imaging device 11 is configured, for example, as shown in FIG. In FIG. 5, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The solid-state imaging device 11 shown in FIG. 5 has an imaging device 91, a signal processing section 27, and a recording section .

The imaging device 91 has the pixel array section 21, the vertical driving section 22, the column processing section 23, the data storage section 24, the horizontal driving section 25, the system control section 26, and the thermometer 29 shown in FIG.

The image sensor 91 supplies (outputs) an image signal obtained by imaging in the pixel array section 21 from the data storage section 24 to the signal processing section 27, and also receives the temperature of the pixel array section 21 measured by the thermometer 29. It is supplied to the signal processing section 27 .

Based on the temperature supplied from the image pickup device 91 and the lookup table recorded in the recording unit 28, the signal processing unit 27 processes the image signal supplied from the image pickup device 91 in step S14 of FIG. Similar signal processing is performed, and the resulting image signal is output to the subsequent stage. Specifically, the signal processing performed by the signal processing unit 27 is, for example, the processing described with reference to FIG.

In this example, since the recording unit 28 is provided outside the imaging element 91, the lookup table recorded in the recording unit 28 can be arbitrarily changed.

<Modification 2 of the first embodiment>
<Configuration example of unit pixel>
Also, the pixel structure of the unit pixel 41 may be a sub-pixel structure having two pixels (sub-pixels) with mutually different sensitivities.

In such a case, the unit pixel 41 is configured, for example, as shown in FIG.

In the example shown in FIG. 6, the unit pixel 41 includes a large pixel 161, a transfer transistor 162, an FD (Floating Diffusion) portion 163, a small pixel 164, a transfer transistor 165, a reset transistor 166, a shutter transistor 167, an amplification transistor 168, and a selection transistor. 169.

The large pixel 161 is a sub-pixel composed of a photodiode that functions as a photoelectric conversion unit, and photoelectrically converts incident light from the outside to generate and store charges (signals) corresponding to the amount of incident light. .

The transfer transistor 162 is turned on or off according to the drive signal supplied from the vertical drive section 22, and when it is turned on (conducting state), the charge accumulated in the large pixel 161 is transferred from the large pixel 161 to the FD section 163. transfer to

The FD portion 163 is a floating diffusion region, and holds charges transferred from the large pixel 161 via the transfer transistor 162 or charges transferred from the small pixel 164 via the transfer transistor 165. (accumulate.

The small pixel 164 is a sub-pixel composed of a photodiode that functions as a photoelectric conversion unit, and photoelectrically converts incident light from the outside to generate and accumulate charges (signals) corresponding to the amount of incident light. .

In particular, the small pixels 164 are smaller than the large pixels 161 here. Also, the large pixel 161 and the small pixel 164 are pixels with different sensitivities, in other words, different quantum efficiencies. In this example, the sensitivity of the large pixel 161 is higher than that of the small pixel 164 .

The transfer transistor 165 is turned on or off according to the drive signal supplied from the vertical drive section 22, and when turned on, transfers the charge accumulated in the small pixel 164 from the small pixel 164 to the FD section 163. .

In addition, not only the transfer transistor 162 and the transfer transistor 165 but also the reset transistor 166 and the amplification transistor 168 are connected to the FD section 163 .

The reset transistor 166 is connected to the power supply V _DD and turned on or off according to the drive signal supplied from the vertical drive section 22 .

When the reset transistor 166 is turned on, the charges accumulated in the FD section 163 are discharged to the power supply _VDD , and the potential of the FD section 163 is reset to a predetermined potential.

The shutter transistor 167 is connected to the power supply V _DD and turned on or off according to the drive signal supplied from the vertical drive section 22 .

When the shutter transistor 167 is turned on, the charge accumulated in the small pixel 164 is discharged to the power supply _VDD , and the potential of the small pixel 164 is reset to a predetermined potential.

A gate electrode of the amplification transistor 168 is connected to the FD section 163, and the amplification transistor 168 amplifies and outputs a signal (charge) transferred from the large pixel 161 or the small pixel 164 to the FD section 163 and held therein. .

That is, the amplification transistor 168 forms a constant current source and a source follower circuit connected via the vertical signal line 43 . The amplification transistor 168 outputs a voltage signal indicating a level corresponding to the charge held in the FD section 163 to the column processing section 23 via the selection transistor 169 and the vertical signal line 43 .

The selection transistor 169 is provided between the source electrode of the amplification transistor 168 and the vertical signal line 43, and is turned on and off in accordance with a drive signal supplied from the vertical driving section 22, thereby connecting the amplification transistor 168 and the vertical signal line. 43 is controlled.

When the unit pixel 41 has the sub-pixel structure as described above, an image obtained by reading out a signal corresponding to the charge obtained by each of the large pixels 161 is the long-accumulation image. On the other hand, an image obtained by reading a signal corresponding to the charge obtained by each of the plurality of small pixels 164 is called a short-term image. At this time, for example, the imaging (exposure) of the long-term image and the imaging of the short-term image are performed at substantially the same timing.

For example, when capturing a long-accumulation image (during readout), charges obtained by photoelectric conversion in the large pixels 161 are transferred to the FD unit 163 and accumulated.

Then, a signal corresponding to the amount of charge from the large pixel 161 accumulated in the FD section 163 is output to the column processing section 23 via the amplification transistor 168, the selection transistor 169, and the vertical signal line 43. The image signal composed of the signal read out from each unit pixel 41 in this way is used as the image signal of the long stored image. In this case, the charge obtained by the small pixel 164 is neither transferred to the FD section 163 nor output to the column processing section 23 .

On the other hand, for example, when capturing a short-term image (reading), the charge obtained by the large pixel 161 is not transferred to the FD section 163, and only the charge obtained by the small pixel 164 is transferred to the FD section 163. , and a signal corresponding to the transferred charge is output to the column processing unit 23 .

The large pixel 161 and the small pixel 164, which have different pixel structures such as size, have different spectral temperature dependencies due to the difference in pixel structure. That is, for example, as shown in FIG. 7, changes in spectral characteristics with respect to changes in temperature of the pixel array section 21 are also different.

In FIG. 7, the vertical axis indicates the quantum efficiency, and the horizontal axis indicates the wavelength of light (incident light).

In particular, in FIG. 7, the portion indicated by arrow Q11 indicates the quantum efficiency of the large pixel 161 at each wavelength, and the portion indicated by arrow Q12 indicates the quantum efficiency of the small pixel 164 at each wavelength. Also, the arrow pointing from the letter "RT" to the letter "125°C" drawn in the portion indicated by the arrow Q11 and the portion indicated by the arrow Q12 indicates that the temperature of the pixel array section 21 is from room temperature to 125°C ( °C) shows the change in quantum efficiency (spectral characteristics).

In the portion indicated by the arrow Q11, the curve L11 indicates the quantum efficiency of the large pixel 161 having the B (blue) color filter, that is, the B component, when the temperature of the pixel array section 21 is room temperature. A curve L12 indicates the quantum efficiency of the B component of the large pixel 161 when the temperature of the pixel array section 21 is 125 degrees.

Comparing the curve L11 and the curve L12, it can be seen that when the temperature rises, the quantum efficiency of the large pixel 161 having the B color filter decreases at most wavelengths.

Similarly, the curve L13 indicates the quantum efficiency of the large pixel 161 having a G (green) color filter, that is, the G component, when the temperature of the pixel array section 21 is room temperature. A curve L14 indicates the G component quantum efficiency of the large pixel 161 when the temperature of the pixel array section 21 is 125 degrees.

Comparing the curves L13 and L14, it can be seen that the quantum efficiency decreases as the temperature rises near the wavelength of 500 nm, but conversely, the quantum efficiency rises (increases) as the temperature rises near the wavelength of 550 nm.

A curve L15 indicates the quantum efficiency of the large pixel 161 having an R (red) color filter, that is, the R component, when the temperature of the pixel array section 21 is room temperature. A curve L16 indicates the quantum efficiency of the R component of the large pixel 161 when the temperature of the pixel array section 21 is 125 degrees.

Comparing the curve L15 and the curve L16, it can be seen that when the temperature rises, the quantum efficiency increases at most wavelengths in the large pixel 161 having the R color filter.

In this way, even with the same large pixel 161, changes in quantum efficiency due to temperature changes differ for each of the R, G, and B color components.

Also, in the portion indicated by the arrow Q12, the curve L21 indicates the quantum efficiency of the B component of the small pixel 164 when the temperature of the pixel array section 21 is room temperature. A curve L22 indicates the quantum efficiency of the B component of the small pixel 164 when the temperature of the pixel array section 21 is 125 degrees (° C.).

Comparing the curve L21 and the curve L22, when the temperature rises, the quantum efficiency of the small pixel 164 having the B color filter decreases at most wavelengths. are different.

Similarly, the curve L23 shows the quantum efficiency of the G component of the small pixel 164 when the temperature of the pixel array section 21 is room temperature, and the curve L24 shows the quantum efficiency when the temperature of the pixel array section 21 is 125 degrees. The quantum efficiency of the G component of the small pixel 164 is shown.

Comparing the curve L23 and the curve L24, it can be seen that the quantum efficiency decreases as the temperature increases near the wavelength of 500 nm, but conversely, the quantum efficiency increases as the temperature increases near the wavelength of 550 nm. Also, it can be seen that the amount of change in quantum efficiency differs between the small pixel 164 and the large pixel 161 even for the same G component.

A curve L25 indicates the quantum efficiency of the R component of the small pixel 164 when the temperature of the pixel array section 21 is room temperature, and a curve L26 indicates the quantum efficiency of the small pixel 164 when the temperature of the pixel array section 21 is 125 degrees. Quantum efficiency of the R component is shown.

Comparing the curve L25 and the curve L26, when the temperature rises, the large pixel 161 having the R color filter increases the quantum efficiency at almost all wavelengths, but the small pixel 164 and the large pixel 161 have the same R component. It can be seen that the amount of change is different.

In this way, even with the same small pixel 164, changes in quantum efficiency due to temperature changes differ for each of the R, G, and B color components. It can also be seen that the large pixel 161 and the small pixel 164 have different spectral characteristics due to temperature changes.

This is because the large pixel 161 and the small pixel 164 differ in pixel structure such as, for example, the size of the photoelectric conversion section (photodiode), that is, the area and shape of the light receiving surface, and the thickness of the photoelectric conversion section.

Therefore, for example, when the quantum efficiency changes in the direction of the arrows drawn in the portions indicated by the arrows Q11 and Q12, it is necessary to increase the quantum efficiency in order to obtain an image signal having desired spectral characteristics. Correction may be performed such that the quantum efficiency, that is, the spectral characteristics, changes in the direction opposite to the arrow representing the change.

Here, the spectral characteristics at a predetermined temperature such as room temperature are assumed to be ideal spectral characteristics, and correction is performed so as to achieve the ideal spectral characteristics.

Specifically, for example, in the large pixel 161 and the small pixel 164, the quantum efficiency of the B component decreases when the temperature rises. good.

In such a case, each of the R component of the short-term image, the G component of the short-term image, the B component of the short-term image, the R component of the long-term image, the G component of the long-term image, and the B component of the long-term image On the other hand, each lookup table of gains and the like prepared in advance for each temperature is used, and signal processing including processing for correcting spectral characteristics is performed on short-term images and long-term images. That is, for each image such as a short-term image and a long-term image, processing for correcting spectral characteristics is performed for each color component individually.

By doing so, for example, even when the temperature of the pixel array section 21 rises, the image signal obtained by the large pixels 161 and the image signals obtained by the small pixels 164 having different pixel structures are different. appropriate correction of spectral characteristics can be performed individually (independently). As a result, an image signal corresponding to desired spectral characteristics, such as spectral characteristics at room temperature, can be obtained, and a desired color reproduction characteristic can be obtained in an image based on the image signal.

In particular, even when HDR synthesis processing is performed using images obtained by each of the large pixel 161 and the small pixel 164 having different structures (sensitivities) provided in the unit pixel 41, the spectral characteristics of each pixel can be Synthesis can be performed by matching and maintaining linearity.

<Modification 3 of the first embodiment>
<Configuration example of solid-state imaging device>
Also, the two images synthesized to obtain the HDR composite image may be obtained by, for example, one imaging device 91 shown in FIG. You may make it image-pickup.

In addition, in FIG. 8, parts corresponding to those in FIG. 5 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

In the example shown in FIG. 8, the solid-state imaging device 201 has an imaging element 91, an imaging element 211, a signal processing section 27, and a recording section .

The imaging device 211 has the same configuration as the imaging device 91, for example. That is, the imaging device 211 has the pixel array section 21, the vertical driving section 22, the column processing section 23, the data storage section 24, the horizontal driving section 25, the system control section 26, and the thermometer 29 shown in FIG. there is

The image sensor 211 supplies (outputs) an image signal obtained by imaging in the pixel array section 21 from the data storage section 24 to the signal processing section 27, and also receives the temperature of the pixel array section 21 measured by the thermometer 29. It is supplied to the signal processing section 27 .

For example, one of the imaging device 91 and the imaging device 211 captures a short-term image, and the other imaging device captures a long-term image.

In such a case, the sensitivities of the unit pixels 41 provided in the pixel array section 21 may be different between the imaging device 91 and the imaging device 211 . That is, for example, one unit pixel 41 of the imaging element 91 and the imaging element 211 may be a pixel such as the large pixel 161, and the other unit pixel 41 may be a pixel such as the small pixel 164. .

In addition, unit pixels 41 having the same sensitivity may be provided in the imaging device 91 and the imaging device 211, and a short-term image and a long-term image may be captured by controlling the exposure time.

3 based on the temperature and image signal supplied from the image sensor 91, the temperature and image signal supplied from the image sensor 211, and the lookup table recorded in the recording unit 28. The same signal processing as in S14 is performed, and the resulting image signal is output to the subsequent stage. Specifically, the signal processing performed by the signal processing unit 27 is, for example, the processing described with reference to FIG.

At this time, for example, in the shading correction for the image signal supplied from the image sensor 91, the temperature of the image sensor 91 and the lookup table for the image sensor 91 are used. Similarly, in the shading correction for the image signal supplied from the image sensor 211, the temperature of the image sensor 211 and the lookup table for the image sensor 211 are used. That is, individual signal processing is performed for each of the image signal obtained by the imaging element 91 and the image signal obtained by the imaging element 211 .

In addition, part of the signal processing including the process of correcting the spectral characteristics performed by the signal processing unit 27, such as shading correction, may be performed by the image sensor 91 or the image sensor 211.

In the solid-state imaging device 201 as described above, even when the outputs of different sensors such as the imaging device 91 and the imaging device 211 are synthesized, the spectral characteristics of the image signal obtained from each sensor are appropriately corrected to achieve the purpose. It is possible to obtain a color reproduction characteristic of

In the example of FIG. 8, an example in which two imaging elements are provided has been described, but the solid-state imaging device 201 may be provided with three or more imaging elements. Even in such a case, signal processing including processing for individually correcting the spectral characteristics is performed on each of the image signals obtained by each of the plurality of imaging elements (pixel array section 21) of three or more.

<Example of application to electronic equipment>
Note that the present technology is not limited to application to solid-state imaging devices. In other words, this technology can be applied to solid-state imaging devices such as digital still cameras and video cameras, portable terminal devices with imaging functions, and copiers that use solid-state imaging devices as image reading units. It is applicable to all electronic devices that use imaging devices. The solid-state imaging device may be formed as a single chip, or may be in a modular form having an imaging function in which an imaging section and a signal processing section or an optical system are packaged together.

FIG. 9 is a block diagram showing a configuration example of an imaging device as an electronic device to which this technology is applied.

An imaging device 501 in FIG. 9 includes an optical unit 511 including a lens group, etc., a solid-state imaging device (imaging device) 512 adopting the configuration of the solid-state imaging device 11 in FIG. Processor) circuit 513 .

The imaging device 501 also includes a frame memory 514 , a display section 515 , a recording section 516 , an operation section 517 and a power supply section 518 . DSP circuit 513 , frame memory 514 , display unit 515 , recording unit 516 , operation unit 517 and power supply unit 518 are interconnected via bus line 519 .

The optical unit 511 captures incident light (image light) from a subject and forms an image on the imaging surface of the solid-state imaging device 512 . The solid-state imaging device 512 converts the amount of incident light imaged on the imaging surface by the optical unit 511 into an electric signal on a pixel-by-pixel basis, and outputs the electric signal as a pixel signal.

The display unit 515 is composed of a thin display such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display, and displays moving images or still images captured by the solid-state imaging device 512 . A recording unit 516 records a moving image or still image captured by the solid-state imaging device 512 in a recording medium such as a hard disk or a semiconductor memory.

The operation unit 517 issues operation commands for various functions of the imaging device 501 under the user's operation. The power supply unit 518 appropriately supplies various power supplies to the DSP circuit 513, the frame memory 514, the display unit 515, the recording unit 516, and the operating unit 517, to these supply targets.

<Usage example of solid-state imaging device>
FIG. 10 is a diagram showing a usage example of the solid-state imaging device 11 described above.

The solid-state imaging device 11 (CMOS image sensor) described above can be used, for example, in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-rays as follows.

・Devices that capture images for viewing purposes, such as digital cameras and mobile devices with camera functions. Devices used for transportation, such as in-vehicle sensors that capture images behind, around, and inside the vehicle, surveillance cameras that monitor running vehicles and roads, and ranging sensors that measure the distance between vehicles. Devices used in home appliances such as TVs, refrigerators, air conditioners, etc., to take pictures and operate devices according to gestures ・Endoscopes, devices that perform angiography by receiving infrared light, etc. equipment used for medical and healthcare purposes ・Equipment used for security purposes, such as surveillance cameras for crime prevention and cameras for personal authentication ・Skin measuring instruments for photographing the skin and photographing the scalp Equipment used for beauty, such as microscopes used for beauty ・Equipment used for sports, such as action cameras and wearable cameras for use in sports ・Cameras, etc. for monitoring the condition of fields and crops , agricultural equipment

<Example of application to a moving body>
In this way, the technology (the present technology) according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure can be realized as a device mounted on any type of moving body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility, airplanes, drones, ships, and robots. may

FIG. 11 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology according to the present disclosure can be applied.

A vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in FIG. 11, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an exterior information detection unit 12030, an interior information detection unit 12040, and an integrated control unit 12050. Also, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/image output unit 12052, and an in-vehicle network I/F (interface) 12053 are illustrated.

The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the driving system control unit 12010 includes a driving force generator for generating driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism to adjust and a brake device to generate braking force of the vehicle.

The body system control unit 12020 controls the operation of various devices equipped on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, winkers or fog lamps. In this case, body system control unit 12020 can receive radio waves transmitted from a portable device that substitutes for a key or signals from various switches. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, etc. of the vehicle.

The vehicle exterior information detection unit 12030 detects information outside the vehicle in which the vehicle control system 12000 is installed. For example, the vehicle exterior information detection unit 12030 is connected with an imaging section 12031 . The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as people, vehicles, obstacles, signs, or characters on the road surface based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of received light. The imaging unit 12031 can output the electric signal as an image, and can also output it as distance measurement information. Also, the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.

The in-vehicle information detection unit 12040 detects in-vehicle information. The in-vehicle information detection unit 12040 is connected to, for example, a driver state detection section 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing off.

The microcomputer 12051 calculates control target values for the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and controls the drive system control unit. A control command can be output to 12010 . For example, the microcomputer 12051 realizes the functions of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane deviation warning. Cooperative control can be performed for the purpose of

In addition, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, etc. based on the information about the vehicle surroundings acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, so that the driver's Cooperative control can be performed for the purpose of autonomous driving, etc., in which vehicles autonomously travel without depending on operation.

Also, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the information detection unit 12030 outside the vehicle. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control aimed at anti-glare such as switching from high beam to low beam. It can be carried out.

The audio/image output unit 12052 transmits at least one of audio and/or image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle. In the example of FIG. 11, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

FIG. 12 is a diagram showing an example of the installation position of the imaging unit 12031. FIG.

In FIG. 12, the vehicle 12100 has imaging units 12101, 12102, 12103, 12104, and 12105 as the imaging unit 12031.

The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose of the vehicle 12100, the side mirrors, the rear bumper, the back door, and the upper part of the windshield in the vehicle interior, for example. An image pickup unit 12101 provided in the front nose and an image pickup unit 12105 provided above the windshield in the passenger compartment mainly acquire images in front of the vehicle 12100 . Imaging units 12102 and 12103 provided in the side mirrors mainly acquire side images of the vehicle 12100 . An imaging unit 12104 provided in the rear bumper or back door mainly acquires an image behind the vehicle 12100 . The forward images acquired by the imaging units 12101 and 12105 are mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 12 shows an example of the imaging range of the imaging units 12101 to 12104. FIG. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively, and the imaging range 12114 The imaging range of an imaging unit 12104 provided in the rear bumper or back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and changes in this distance over time (relative velocity with respect to the vehicle 12100). , it is possible to extract, as the preceding vehicle, the closest three-dimensional object on the course of the vehicle 12100, which runs at a predetermined speed (for example, 0 km/h or more) in substantially the same direction as the vehicle 12100. can. Furthermore, the microcomputer 12051 can set the inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including following stop control) and automatic acceleration control (including following start control). In this way, cooperative control can be performed for the purpose of automatic driving in which the vehicle runs autonomously without relying on the operation of the driver.

For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 converts three-dimensional object data related to three-dimensional objects to other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into those that are visible to the driver of the vehicle 12100 and those that are difficult to see. Then, the microcomputer 12051 judges the collision risk indicating the degree of danger of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, an audio speaker 12061 and a display unit 12062 are displayed. By outputting an alarm to the driver via the drive system control unit 12010 and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be performed.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not the pedestrian exists in the captured images of the imaging units 12101 to 12104 . Such recognition of a pedestrian is performed by, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and performing pattern matching processing on a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. This is done by a procedure that determines When the microcomputer 12051 determines that a pedestrian exists in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a rectangular outline for emphasis to the recognized pedestrian. is superimposed on the display unit 12062 . Also, the audio/image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 and the like among the configurations described above. Specifically, for example, the solid-state imaging device 11 shown in FIG. 1 can be used as the imaging unit 12031, and desired spectral characteristics can be obtained.

Note that this technology is not limited to application to solid-state imaging devices that detect the distribution of the amount of incident visible light and capture an image. In a broad sense, it can be applied to solid-state imaging devices (physical quantity distribution detectors) such as fingerprint detection sensors that detect the distribution of other physical quantities such as pressure and capacitance and capture images. be.

In addition, the present technology is applicable not only to solid-state imaging devices but also to semiconductor devices in general having other semiconductor integrated circuits.

Embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

For example, a form obtained by combining all or part of the multiple embodiments described above can be adopted.

Also, the effects described in this specification are merely examples and are not limited, and there may be effects other than those described in this specification.

Furthermore, this technology can also be configured as follows.

(1)
A semiconductor device, comprising: a signal processing section that performs signal processing for correcting spectral characteristics based on a temperature of a pixel array section, on an image signal obtained by imaging a pixel array section having a plurality of unit pixels.
(2)
The semiconductor device according to (1), wherein the signal processing unit performs the signal processing for each color component of the image signal.
(3)
The semiconductor device is
the pixel array section;
a thermometer provided near the pixel array section for measuring the temperature of the pixel array section;
The semiconductor device according to (1) or (2), which is an imaging element including the signal processing unit.
(4)
The semiconductor device according to (3), wherein the thermometer is arranged inside a substrate on which the pixel array section is formed.
(5)
The semiconductor device according to (3) or (4), wherein the thermometer is arranged near the pixel array section and near a circuit provided around the pixel array section. .
(6)
The semiconductor device according to any one of (3) to (5), including a plurality of the thermometers arranged at mutually different positions near the pixel array section.
(7)
The semiconductor device according to any one of (3) to (5), further comprising a plurality of the thermometers arranged adjacent to each other in the vicinity of the pixel array section.
(8)
The semiconductor device according to any one of (3) to (7), wherein the pixel array section and the signal processing section are formed on the same substrate.
(9)
further comprising a recording unit for recording a table for obtaining coefficients determined for the temperature;
The signal processing unit according to any one of (3) to (8), wherein the signal processing is performed based on the coefficient corresponding to the temperature measured by the thermometer, which is obtained from the table. semiconductor device.
(10)
the unit pixel has a first pixel and a second pixel having pixel structures different from each other;
The signal processing unit separately performs the signal processing on the image signals obtained by the plurality of first pixels and the image signals obtained by the plurality of second pixels (1) to ( 9) The semiconductor device according to any one of items.
(11)
any one of (1) to (9), wherein a plurality of the pixel array units are provided, and the signal processing is performed individually for each of the image signals obtained by the plurality of the pixel array units. 1. The semiconductor device according to claim 1.
(12)
The semiconductor device according to any one of (1) to (11), wherein the signal processing is shading correction, HDR synthesis processing, or white balance adjustment processing, including processing for correcting spectral characteristics.
(13)
The semiconductor device according to any one of (1) to (12), wherein the signal processing section corrects spectral characteristics when the temperature is outside a predetermined range.
(14)
A semiconductor device
A signal processing method, comprising: performing signal processing for correcting spectral characteristics based on a temperature of a pixel array section, on an image signal obtained by imaging a pixel array section having a plurality of unit pixels.
(15)
A program for causing a computer to execute processing including a step of performing signal processing for correcting spectral characteristics based on the temperature of the pixel array section on image signals obtained by imaging with a pixel array section having a plurality of unit pixels.

11 solid-state imaging device, 21 pixel array unit, 23 column processing unit, 27 signal processing unit, 28 recording unit, 29-1, 29-2, 29 thermometer, 161 large pixels, 164 small pixels

Claims (15)

1. A semiconductor device, comprising: a signal processing section that performs signal processing for correcting spectral characteristics based on a temperature of a pixel array section, on an image signal obtained by imaging a pixel array section having a plurality of unit pixels.
2. The semiconductor device according to claim 1, wherein the signal processing section performs the signal processing for each color component of the image signal.
3. The semiconductor device is
   the pixel array section;
   a thermometer provided near the pixel array section for measuring the temperature of the pixel array section;
   2. The semiconductor device according to claim 1, which is an imaging device including the signal processing unit.
4. 4. The semiconductor device according to claim 3, wherein said thermometer is arranged inside a substrate on which said pixel array section is formed.
5. 4. The semiconductor device according to claim 3, wherein the thermometer is arranged near the pixel array section and near a circuit provided around the pixel array section.
6. 4. The semiconductor device according to claim 3, comprising a plurality of said thermometers arranged at mutually different positions in the vicinity of said pixel array section.
7. 4. The semiconductor device according to claim 3, comprising a plurality of said thermometers arranged at positions adjacent to each other in the vicinity of said pixel array section.
8. The semiconductor device according to claim 3, wherein the pixel array section and the signal processing section are formed on the same substrate.
9. further comprising a recording unit for recording a table for obtaining coefficients determined for the temperature;
   4. The semiconductor device according to claim 3, wherein the signal processing section performs the signal processing based on the coefficient obtained from the table and corresponding to the temperature measured by the thermometer.
10. the unit pixel has a first pixel and a second pixel having pixel structures different from each other;
    2. The signal processing unit according to claim 1, wherein the signal processing section separately performs the signal processing on image signals obtained from the plurality of first pixels and image signals obtained from the plurality of second pixels. semiconductor equipment.
11. 2. The semiconductor device according to claim 1, comprising a plurality of said pixel array sections, wherein said signal processing is individually performed on each of image signals obtained from said plurality of said pixel array sections.
12. 2. The semiconductor device according to claim 1, wherein the signal processing is shading correction, HDR synthesis processing, or white balance adjustment processing including processing for correcting spectral characteristics.
13. 2. The semiconductor device according to claim 1, wherein said signal processing section corrects spectral characteristics when said temperature is outside a predetermined range.
14. A semiconductor device
    A signal processing method, comprising: performing signal processing for correcting spectral characteristics based on a temperature of a pixel array section, on an image signal obtained by imaging a pixel array section having a plurality of unit pixels.
15. A program for causing a computer to execute processing including a step of performing signal processing for correcting spectral characteristics based on the temperature of the pixel array section on image signals obtained by imaging with a pixel array section having a plurality of unit pixels.

Images