JP7154776B2 - Semiconductor devices, imaging systems, moving objects - Google Patents

Description

The present invention relates to semiconductor devices, imaging systems, and moving bodies.

In recent years, various semiconductor devices have been used. As the operation period of a semiconductor device becomes longer, the function of the semiconductor device may deteriorate.

For example, the technology described in Patent Document 1 describes a semiconductor device including a life prediction unit that acquires the degree of deterioration of a functional unit included in the semiconductor device and predicts the life of the semiconductor device from the degree of deterioration. Furthermore, in Patent Document 1, the power supply voltage Vdd is increased when the threshold voltage increases due to deterioration of the semiconductor element. This is said to extend the life of the functional unit.

JP 2017-173242 A

In the technique described in Patent Document 1, driving is performed by increasing the power supply voltage in order to prolong the life of the semiconductor device. However, increasing the power supply voltage sometimes shortens the life of the semiconductor device, and is not the most suitable driving method for extending the life of the semiconductor device.

The present invention is the result of a study on driving for extending the remaining life of a semiconductor device.

The present invention has been made in view of the above problems, and one aspect of the invention includes a semiconductor element that performs signal processing, a drive control unit that controls driving of the semiconductor element, and a remaining life of the semiconductor element. and a life acquisition unit for acquiring remaining life information indicating the remaining life information, and when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition, and the remaining life information indicates a second length that is shorter than the first length, the drive control unit is under the condition that the processing amount of the signal processing is lower than when the semiconductor element is driven under the first condition. and driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition, and the drive control unit controls the semiconductor element under the first condition. When the device is driven, the lifespan obtaining unit obtains the remaining lifespan information at a first interval, and when the drive control unit drives the semiconductor device under the second condition, the lifespan is obtained. The part acquires the remaining life information at a second interval shorter than the first interval .

The present invention can extend the remaining life of a semiconductor device.

A diagram showing a configuration of a semiconductor device Flowchart showing the operation of a semiconductor device Diagram showing the relationship between operating time and life of a semiconductor device The figure which showed the structure of an imaging device Pixel equivalent circuit diagram Block diagram of the column circuit Diagram showing the operation of the imaging device The figure which showed the structure of an imaging device The figure which showed the structure of an imaging device A diagram showing the configuration of an imaging system Diagram showing the configuration of the moving body Diagram showing the operation of the imaging system

Each embodiment will be described below with reference to the drawings. In the following description, transistors are assumed to be N-type transistors unless otherwise specified. However, the embodiments described below are not limited to N-type transistors, and P-type transistors may be used as appropriate. In that case, the voltages of the gate, source and drain of the transistor can be appropriately changed from the description in the embodiments. For example, in the case of a transistor operated as a switch, the low level and high level of the voltage supplied to the gate may be reversed with respect to the description in the embodiments.

(Example 1)
FIG. 1 is a diagram showing an example of the configuration of a semiconductor device according to this embodiment. A semiconductor device 150 has a semiconductor element 100 and a life prediction unit 109 . The semiconductor device 100 is a circuit in which many functions such as light reception, light emission, AD conversion processing, signal processing, communication processing, storage section, and detection section are integrated. The configuration and operation of the semiconductor device 100 will be described below.

The first circuit block 101, the second circuit block 102, and the Nth circuit block 103 are circuits having many functions such as light reception, light emission, AD conversion processing, signal processing, communication processing, storage section, and detection section. .

The voltage control unit 104 supplies the first circuit block 101, the second circuit block 102, and the Nth circuit block 103 with the voltage value set by the signal from the drive mode selection unit 105, which is the drive control unit.

When the voltage value supplied by the voltage control unit 104 is zero, all or part of the first circuit block 101, the second circuit block 102, and the Nth circuit block 103 are turned off.

The frequency control unit 106 controls the signal generation period of the signal generation unit 107 that generates the respective control signals supplied to the first circuit block 101, the second circuit block 102, and the Nth circuit block 103 to the drive mode generation unit. changed by a signal from 105;

Remaining life information is input to the driving mode selection unit 105 from a life prediction unit 109 which is a life acquisition unit (to be described later) via a life information reception unit 108 . The voltage control section 104 and the frequency control section 106 are controlled according to the information. The drive mode generator 105 can also generate a control signal capable of stopping all or part of the first circuit block 101, the second circuit block 102, and the Nth circuit block 103. FIG.

The life prediction unit 109 executes life prediction of the semiconductor device 100 based on the latest operating state of the semiconductor device 100 or history information related to the operating state, and obtains remaining life information of the semiconductor device 100 . That is, the life prediction unit 109 is a life acquisition unit that acquires life information. Here, the operating state means the operating state of the semiconductor device 100, such as operating voltage, operating frequency, or the like, or the state resulting from the operation, such as temperature.

Next, an example of a life prediction method of the semiconductor element 100 by the life prediction unit 109 will be described. In general, the life of a semiconductor device is limited by deterioration in the performance of transistors, wiring, oxide films, and the like that constitute the semiconductor device over time. Major performance degradation mechanisms of semiconductor devices include electromigration (EM) and time dependent dielectric breakdown (TDDB). Other performance degradation mechanisms include characteristic fluctuations due to hot carrier injection (HC) and negative bias temperature instability (NBTI).

The remaining life time LT (EM) due to EM as a performance degradation factor is calculated by the following formula (1).
LT(EM)=A×J− ⁿ ×exp(Ea/kT) (1)

Here, A is a coefficient dependent on the manufacturing process, Ea is the activation energy, J is the current density of the wiring, n is the current acceleration factor, k is the Boltzmann constant, and T is the wiring temperature.

The remaining life time LT (TDDB) due to TDDB as a performance degradation factor is calculated by the following equation (2).
LT(TDDB)=A×exp(−γVg×Vg)×exp(Ea/kT) (2)

Here, Vg is the voltage applied to the gate of the transistor, and γVg is the voltage acceleration factor.

The remaining life time LT (HC) due to HC being the performance deterioration factor is calculated by the following equation (3).
LT(HC)=A×Isub− ^m ×exp(Ea/kT) (3)

Here, Isub is the maximum substrate current flowing through the semiconductor substrate, and m is a substrate current dependent coefficient.

Remaining life time LT (NBTI) due to NBTI as a factor of performance deterioration of the semiconductor device 100 is calculated by the following equation (4).
LT(NBTI)=A× ^Vgn ×exp(Ea/kT) (4)

The life prediction unit 109 measures all or part of the coefficients of formulas (1) to (4) to calculate the remaining life of the semiconductor device 100 . Note that the life prediction unit 109 may be inside the semiconductor device 100 .

An example of the operation of the present invention will be described below with reference to the flow chart of FIG. 2 and the timing chart of FIG. First, an example of the operation corresponding to the timing chart of FIG. 3(a) will be described. In FIG. 3A, the solid line represents the relationship between the operating time of the semiconductor device 100 and the remaining life of the semiconductor device 100 .

In S<b>200 , the drive mode selection unit 105 acquires the first life threshold L<b>1 and the second life threshold L<b>2 of the semiconductor device 100 . The first life threshold L1 and the second life threshold L2 define thresholds for the remaining life of the semiconductor device 100, and L1>L2>0.

In S201, the life prediction unit 109 calculates the remaining life Ti of the semiconductor device 100 at time i.

In S202, the driving mode selection unit 105 compares the first life threshold value L1 with the remaining life Ti at time i. If Ti>L1, the process proceeds to S203. proceed.

FIG. 3A shows the case of Ti>L1.

In S203, the process of continuing the operation of the semiconductor device 100 is performed for an arbitrary period of time Δt1, and then the process returns to the process S201.

The above flow shows that the remaining life of the semiconductor device 100 is longer than the first life threshold value L1, and there is a margin in the remaining life of the semiconductor device 100. Therefore, the driving state of the semiconductor device 100 is changed. not.

Next, the case of FIG. 3(b), in which the operating time has elapsed from FIG. 3(a), will be described.

In FIG. 3B, the relationship between the operating time of the semiconductor device 100 and the remaining life of the semiconductor device 100 is represented by a solid line. Also, the predicted remaining life of the semiconductor device 100 with respect to the operating time of the semiconductor device 100 predicted by the life prediction unit 109 at the time i is indicated by a dashed line.

FIG. 3B shows the case of Ti≦L1.

In S204, the life prediction unit 109 predicts the remaining life Lt of the semiconductor device 100 when the set operation time t has elapsed. The set operation time is, for example, the time required for the semiconductor device 100 to reach its estimated lifetime. If the semiconductor device 100 operates beyond this time, the operation of the semiconductor device 100 is not guaranteed.

In S205, it is compared whether or not the remaining life Lt of the semiconductor device 100 after the set operation time t has elapsed is greater than 0. If Lt>0, the process proceeds to S203, and if Lt≦0, the process proceeds. Proceed to S206. Note that FIG. 3B shows the case of Lt>0.

The case of Lt>0 indicates that the semiconductor device 100 does not reach the end of its life even if the normal drive mode is continued until the set operation time t. Therefore, the drive mode selector 105 does not change the drive mode of the semiconductor element 100 .

The process proceeds to S203, performs the process of continuing the operation of the semiconductor element 100 in the normal drive mode for an arbitrary period Δt1, and then returns to S201.

As described above, in the case of FIG. 3B, the normal drive mode is continued until the drive mode setting operation time t of the semiconductor device 100 .

Next, the case of FIG.3(c) is demonstrated.

In the case of FIG. 3C, if the operation in the normal drive mode is continued after time i, the life of the semiconductor element 100 is reached before the set operation time t is reached. That is, in the process of S205, Lt>0 is determined as No.

In FIG. 3C, the solid line represents the relationship between the operating time of the semiconductor device 100 and the remaining life of the semiconductor device 100 . Also, the predicted remaining life of the semiconductor device 100 with respect to the operating time of the semiconductor device 100 predicted by the life prediction unit 109 at the time i is indicated by a dashed line. At time i, the driving state of the semiconductor device 100 is changed, and the relationship between the operating time of the semiconductor device 100 in the changed driving state and the remaining life of the semiconductor device 100 is represented by a dashed line.

In S205 of FIG. 2, it is determined as No in the case of FIG. 3(c). Therefore, the process proceeds to S206.

In S206 of FIG. 2, the drive mode selection unit 105 controls the operation of either the voltage control unit 104 or the frequency control unit 106 so that the remaining life of the semiconductor device 100 becomes longer than the remaining life at time i. select a life-prolonging drive mode. The life-prolonging drive mode is a condition under which the throughput of signal processing is lower than when the semiconductor device 100 is driven in the normal drive mode. In this drive mode, the remaining life of the semiconductor device 100 is longer than when the semiconductor device 100 is driven in the normal drive mode. That is, the first condition for driving the semiconductor device 100 by the drive control unit is the normal drive mode. The semiconductor element 100 is driven under the first condition, and the amount of signal processing is lower than when the semiconductor element 100 is driven under the first condition. The second condition is the life extension drive mode.

Next, an example of a method for selecting the life-prolonging drive mode of the semiconductor device 100 by the drive mode selection unit 105 will be described. As described in the formulas (1) to (4) of Embodiment 1, the lifetime of a semiconductor device depends on the current density J of the wiring, the wiring temperature T, the voltage Vg applied to the gate of the transistor, and the maximum substrate current Isub flowing through the semiconductor substrate. Limited by The drive mode selection unit 105 controls both or one of the voltage control unit 104 and the frequency control unit 106 so that the remaining life of the semiconductor device 100 is lengthened. Specifically, all or some of the current density J, wiring temperature T, voltage Vg applied to the gate of the transistor, and maximum substrate current Isub flowing through the semiconductor substrate are smaller than in the normal drive mode. change the operating state to It can also be said that this change in operating state is a change that suppresses the drive capability for driving the semiconductor device 100 . For example, the voltage control unit 104 reduces the absolute value of each power supply voltage supplied to the first circuit block 101, the second circuit block 102, and the Nth circuit block 103 (typically, the power supply voltage is reduced). ). Thereby, the current density J, the wiring temperature T, and the voltage Vg can be lowered. In addition, the wiring temperature T and the substrate current Isub can be lowered by delaying the signal generation period of the signal generation section 107 using the frequency control section 106 . At this time, the drive mode selection unit 105 controls the voltage control unit 104 and the frequency control unit 106 within a range in which the operation of the semiconductor device 100 can be continued.

The normal drive mode is a mode in which the semiconductor device 100 is driven under the first condition, as described above. As described above, the life-prolonging drive mode is a mode in which the semiconductor device 100 is driven under the second condition to prolong the life of the semiconductor device 100 as compared with the case where the semiconductor device 100 is continuously driven in the normal drive mode.

By changing to the life-prolonging drive mode, the amount of signal processing of the semiconductor device 100 may be lowered as compared with the normal drive mode, such as stopping some functions and lowering the operation speed of the semiconductor device 100 . However, in the life-extending drive mode, the operation of the semiconductor device 100 can be continued, and the load applied to the circuit of the semiconductor device 100 can be reduced compared to the normal drive mode. can be extended longer than

In the life extension drive mode, the throughput of signal processing of the semiconductor device 100 is lower than in the normal drive mode. The processing amount of this signal processing can be said to be, for example, the signal amount that the semiconductor device 100 performs signal processing per unit time. This example includes an example in which one or more of the operating frequency of the semiconductor device 100, the drive voltage (power supply voltage), and the amount of current supplied by the current source are made smaller in the life extension drive mode than in the normal drive mode. As another example, the normal driving mode performs signal processing in the first period. In the life extension drive mode, signal processing is performed in a second period shorter than the first period, and intermittent driving is performed in which signal processing is not performed in a period included in the first period and not included in the second period. Alternatively, driving that reduces the amount of signal processing compared to the second period is also included. The example of FIG. 3(c) shows a case where the remaining life of the semiconductor device 100 operating in the life extension drive mode is longer than the second life threshold L2 and further operation is possible in the normal drive mode.

In S207, the life prediction unit 109 calculates the remaining life Lj of the semiconductor device 100 at time j.

In S208, the drive mode selection unit 105 compares the second life threshold value L2 with the remaining life Lj at time j. If Lj>L2, the process proceeds to S209. . FIG. 3C illustrates the case of Lj>L2.

The state of Lj>L2 indicates that the remaining life of the semiconductor device 100 operating in the life extension drive mode is shorter than the first life threshold L1 but longer than the second life threshold L2. . Therefore, in the life-prolonging drive mode, there is a sufficient margin for the remaining life of the semiconductor device 100 with respect to the set operation duration. Therefore, even if the semiconductor device 100 is operated for a further period of time in the normal drive mode instead of the life extension drive mode, the semiconductor device 100 can continue to operate until the set operation continuation period. Therefore, the drive state of the semiconductor device 100 is returned to the normal drive mode before changing to the life extension drive mode. Then, the process proceeds to S203, and the operation of the semiconductor device 100 is continued in the normal drive mode for an arbitrary period Δt1. After that, the process returns to S201.

Next, the example of FIG.3(d) is demonstrated.

In FIG. 3D, the predicted remaining life of the semiconductor device 100 with respect to the operating time of the semiconductor device 100 predicted by the life prediction unit 109 at time i is indicated by a dashed line. At time i, the driving state of the semiconductor device 100 is changed, and the relationship between the operating time of the semiconductor device 100 in the changed driving state and the remaining life of the semiconductor device 100 is represented by a dashed line.

In S208, the drive mode selection unit 105 compares the second life threshold value L2 with the remaining life Lj at time j. FIG. 3D shows the case of Li≦L2.

In S210, the life prediction unit 109 calculates the remaining life Lk of the semiconductor device 100 at time k.

In S211, the drive mode selection unit 105 compares the second life threshold value L2 with the remaining life Lk at time k, and if Lk>0, the process proceeds to S212. On the other hand, if Lk≦0, the process proceeds to S213. In this embodiment, a case where Lk>0 is satisfied and the process proceeds to S212 will be described.

A state of Lk>0 is a case where the remaining life of the semiconductor device 100 operating in the life extension driving mode is shorter than the first life threshold L2. Although the remaining life of the semiconductor device 100 is close to 0, it is longer than 0. Therefore, the process proceeds to S212 to continue the operation of the semiconductor device 100 for an arbitrary period Δt2. The process then returns to S210. At this time, since the remaining life of the semiconductor element 100 is shorter than the second life threshold L2, the period Δt2, which is the interval for determining the remaining life Lk, is the remaining life Li when the remaining life is longer than the first life threshold L1. is shorter than the period .DELTA.t1, which is the interval for obtaining . This makes it possible to accurately detect the time when the normal operation of the semiconductor device 100 is no longer guaranteed.

Next, the example of FIG.3(e) is demonstrated.

FIG. 3(e) illustrates the case of Lk≦0.

The state of Lk≦0 indicates that the remaining life Lk of the semiconductor device 100 is 0 or less and the operation is not guaranteed. The process advances to S213 to inform the outside that the semiconductor device 100 is in a state where the operation is not guaranteed. As a method of this notification, there is an example of notifying the user of the semiconductor device 100 through a display not shown in FIG. As another example, there is an example of notifying a monitoring system that monitors the semiconductor device 100 via a communication line. After that, the process proceeds to S214, and the operation of the semiconductor device 100 is stopped.

In the case of FIG. 3E, the remaining life of the semiconductor element 100 is 0 or less at time k, and the operation of the semiconductor element 100 is not guaranteed. Therefore, the outside of the semiconductor device 100 is notified that the operation of the semiconductor device 100 is not guaranteed. The operation of the semiconductor device 100 is then stopped.

Therefore, although the remaining life of the semiconductor device 100 is 0 or less, it is possible to prevent the operation from being continued in a state where the operation is not guaranteed.

In this manner, the semiconductor device of this embodiment changes the drive state based on the remaining life information. Thereby, the semiconductor device 100 can be driven according to the length of the remaining life. Further, when the remaining life of the semiconductor device 100 falls below the first life threshold, the normal drive mode is changed to the life extension drive mode. Thereby, the life of the semiconductor device 100 can be extended while the operation of the semiconductor device 100 is continued. Further, when the remaining life of the semiconductor device 100 is below the second life threshold, the interval at which remaining life information is acquired is made shorter than when the remaining life exceeds the first life threshold. Thereby, the remaining life of the semiconductor device 100 can be accurately obtained. Further, when the remaining life of the semiconductor device 100 is 0 or less, the outside of the semiconductor device 100 is notified. As a result, it is possible to prevent the operation of the semiconductor device 100 from being continued without the operation being guaranteed.

In addition, in the present embodiment, the driving mode of the semiconductor element 100 is changed depending on whether or not the remaining life exceeds the first life threshold. It is not limited to this example, but when the remaining life indicates the first length, the normal drive mode is set, and when the remaining life indicates the second length shorter than the first length, the life is extended. A drive mode may be used.

Also, in this embodiment, an example in which the semiconductor device has the life prediction unit 109 has been described, but the present invention is not limited to this example. The semiconductor device may have a time acquisition unit that acquires time information that indicates the elapsed operating time of the semiconductor element 100 . In this case, when the time information indicates the first length, the semiconductor device 100 is driven under the first condition. Then, when the time information indicates a second length longer than the first length, the semiconductor device 100 is driven under the second condition. The second condition is a condition in which the throughput of signal processing is lower than when the semiconductor element is driven under the first condition, and the remaining life of the semiconductor element is lower than when the semiconductor element is driven under the first condition. is the long condition.

(Example 2)
In this embodiment, a case where an imaging device is used as the semiconductor element 100 described in the first embodiment will be described.

The imaging device 450 shown in FIG. 4 has an imaging device and a life prediction unit 109 . The imaging device 400 is a CMOS sensor. It comprises a pixel array 402 in which pixels 401 are arranged over multiple rows and multiple columns, a vertical scanning circuit 403 , a vertical output line 404 and a column circuit 405 . The imaging device 400 also includes a reference signal generation circuit 406 , a storage section 407 , a counter circuit 408 , a horizontal scanning circuit 409 , a signal processing circuit 410 , an imaging mode control section 411 and a life prediction section 109 .

Here, the configuration of the pixel 401 will be described with reference to FIG.

Each pixel 401 has a photodiode 501a, 501b. Photodiodes 501a and 501b of one pixel 401 receive light that has passed through one microlens (not shown). Stated another way, the photodiodes 501a, 501b receive light from different regions of the exit pupil of the optical system that directs light to the CMOS sensor. Therefore, phase-difference focus detection can be performed by using signals based on charges photoelectrically converted by the photodiodes 501a and 501b.

An image can be generated using a signal corresponding to the sum of the charge photoelectrically converted by the photodiode 501a and the charge photoelectrically converted by the photodiode 501b.

The photodiodes 501a and 501b accumulate charges corresponding to incident light. The vertical scanning circuit 403 shown in FIG. 4 performs vertical scanning for reading out signals from the pixels 401 arranged over the rows on a row-by-row basis. The charge accumulated in the photodiode 501a is transferred to the floating diffusion portion (FD portion) 503 by setting the signal txa output to the transfer gate 502a to High level.

Also, the vertical scanning circuit 403 sets the signal txb output to the transfer gate 502b to High level, whereby the charge accumulated in the photodiode 501b is transferred to the FD section 503. FIG.

In the FD section 503 , the charges transferred from the photodiodes 501 a and 501 b are converted into voltage based on the capacitance of the FD section 503 . The FD section 503 is connected to the gate of the amplification transistor 504 . The amplification transistor 504 is connected to the vertical output line 404 via the selection transistor 506 .

The pixel 401 has a reset transistor 505 connected to the FD section 503 . The reset transistor 505 resets the FD section 503 when the vertical scanning circuit 403 sets the signal res to High level. When performing photodiode reset for resetting the charges of the photodiodes 501a and 501b, the vertical scanning circuit sets the signals txa and txb to high level while the signal res is set to high level.

Signals res, txa, txb, and sel supplied from the vertical scanning circuit 403 are commonly input to pixels 401 in one row. The output vout of each pixel 401 is input to the corresponding column circuit 405 via the vertical output line 404 arranged corresponding to the column in which the pixels 401 are arranged.

Here, the configuration of the column circuit 405 arranged for each column will be described with reference to FIG.

FIG. 6 shows a configuration of column circuit 405. Referring to FIG. Column circuit 405 has current source 601 , switch 602 , differential amplifier 603 , switch 604 , comparator 605 and switch 606 .

A switch 602 is a switch that switches the current source 601 on and off. A switch 604 is a switch for switching on and off of the differential amplifier 603 . A switch 606 is a switch for switching on and off of the comparator 605 .

When the switch 602 is on, the vertical scanning circuit 403 sets the signal sel to be supplied to the pixels 401 in the row to be read to High level. As a result, current flows between the amplification transistor 504 and the current source 601 via the selection transistor 506 of the pixel 401 in the row to be read. Thereby, the amplification transistor 504 performs source follower operation. As a result, a signal (pixel signal) based on the voltage of the FD section 503 is output to the vertical output line 404 .

When switch 604 is on, differential amplifier 603 is in operation. In this case, the differential amplifier circuit 603 amplifies the pixel signal output from the pixel 401 to the vertical output line 404 and outputs an amplified signal to the comparator 605 .

When switch 605 is on, comparator 605 is active. In this case, comparator 605 compares the voltage of the amplified signal output from differential amplifier 603 with the voltage of the ramp signal supplied from reference signal generating circuit 406 . Comparator 605 then outputs the result of this comparison as signal Cout to storage unit 407 shown in FIG. The signal level of the signal Cout changes when the magnitude relationship of the signal voltages is inverted. A count signal is input from the counter circuit 408 to the storage unit 407 . This count signal changes the count value corresponding to the start timing of the voltage change of the ramp signal supplied by the reference signal generation circuit 406 . Then, the storage unit 407 latches the count value based on the timing when the signal level of the signal Cout changes. Thereby, the storage unit 407 can obtain digital data corresponding to the signal level of the amplified signal. In this manner, AD conversion of pixel signals is performed by the column circuit 405 and the storage unit 407 .

The digital data stored in the storage unit 407 is sequentially transferred to the signal processing circuit 410 column by column by the horizontal scanning circuit 409 . A series of operations related to reading out pixel signals from these pixels is performed while the vertical scanning circuit 403 selects a pixel row of the pixel array 402 .

FIG. 7 is a timing chart showing the operation of the imaging device shown in FIG. FIG. 7(a) is a timing chart for imaging operation. FIG. 7(b) is a timing chart when the focus detection operation and the imaging operation are performed. Pixel signals are read out from the pixels 401 included in some of the rows of pixels 401 arranged in the pixel array 402 by the focus detection operation and the imaging operation shown in FIG. 7B. Pixel signals are read from the pixels 401 included in some other rows by the imaging operation shown in FIG. 7A. This part of the rows and the other part of the rows are assigned according to the area for focus detection. As another example, the pixels 401 in all rows perform the operation shown in FIG. 7A in a certain frame. Then, in another frame, the operation of FIG. 7(b) may be performed. The frame in this case can be, for example, a period from when the vertical scanning circuit 403 starts vertical scanning to when it starts the next vertical scanning. Typically, a control circuit (not shown) sets the signal level of a vertical synchronization signal for instructing the start of vertical scanning of the vertical scanning circuit 403 to High level, and then sets the signal level of the vertical synchronization signal to High level. can be a period of As another example, after starting to read out signals corresponding to one image generated by the signal processing unit from the pixel array 402, signals corresponding to the next one image are read out from the pixel array 402. It can also be a period until reading is started. For example, suppose that a moving image is shot at 60 fps (fps is an abbreviation for frame per second). In this example, the signal corresponding to the image of one frame out of 60 frames is started to be read from the pixel array 402, and then the signal corresponding to the image of the next one frame is read out from the pixel array 402. It can be a period until the start.

First, the operation of FIG. 7(a) will be described.

At time ta1, the vertical scanning circuit 403 sets the signal sel supplied to the row of the pixels 401 that perform the imaging operation to High level. This turns on the selection transistor 506 of the pixel 401 .

After that, at time ta2, the vertical scanning circuit 403 turns the signal res to Low level to turn off the reset transistor 505 . As a result, the reset of the FD unit 503 is released.

The amplification transistor 504 outputs a noise signal (N signal) to the vertical output line 404 via the selection transistor 506 based on the voltage of the FD section 503 whose reset has been released. The N signal output to the vertical output line 404 is converted into digital data (N data) by AD conversion by the column circuit 405 and storage unit 407 .

At time ta3, the vertical scanning circuit 403 changes the signals txa and txb to High level and then to Low level. By this operation, the charge accumulated by the photodiode 501 a based on the incident light and the charge accumulated by the photodiode 501 b based on the incident light are transferred to the FD section 503 . Therefore, the voltage of the FD section 503 becomes a voltage corresponding to the sum of the charge of the photodiode 501 a and the charge of the photodiode 501 .

The amplification transistor 504 outputs a signal based on the voltage of the FD section 503 corresponding to the sum of the charge of the photodiode 501 a and the charge of the photodiode 501 to the vertical output line 404 via the selection transistor 506 . A signal output from the amplifying transistor 504 will be described. If the voltage of the FD section 503 is the voltage based on the charge accumulated by the photodiode 501a based on the incident light, the signal output by the amplification transistor 504 is the A+N signal (the sum of the A signal and the N signal). do. When the voltage of the FD section 503 is the voltage based on the charge accumulated by the photodiode 501b based on incident light, the signal output by the amplification transistor 504 is assumed to be the B+N signal (the sum of the B signal and the N signal). In such a case, the signal based on the voltage of the FD unit 503, which corresponds to the sum of the charge of the photodiode 501a and the charge of the photodiode 501b, output by the amplification transistor 504 is the sum of the A+N signal and the B+N signal. It corresponds to a signal. Therefore, the signal output from this amplifying transistor 504 is expressed as the A+B+N signal. The A+B+N signal output to the vertical output line 404 is converted into digital data (A+B+N data) by AD conversion by the column circuit 405 and storage unit 407 .

At time ta5, the vertical scanning circuit 403 changes the signal res to High level. Thereby, the voltage of the FD section 503 is reset.

Then, at time ta6, the vertical scanning circuit 403 changes the signal sel to Low level. This completes the selection of the pixels 401 in one row. After that, the vertical scanning circuit 403 sets the signal sel of the pixels 401 in the next readout row to High level.

The N data and A+B+N data held by the storage unit 407 are sequentially read from the storage unit 407 to the signal processing circuit 410 column by column by the horizontal scanning circuit 409 . The signal processing circuit 410 outputs A+B data, which is a difference signal between the A+B+N data and the N data.

Next, the operation of FIG. 7(b) will be described.

The operation at time tb1 is the same as the operation at time ta1 in FIG. 7(a).

The operation at time tb2 is the same as the operation at time ta2 in FIG. 7(a).

At time tb3, the vertical scanning circuit 403 changes the signal txa to High level and then to Low level. As a result, the charge accumulated by the photodiode 501 a based on incident light is transferred to the FD section 503 . Amplification transistor 504 outputs the A+N signal to vertical output line 404 via selection transistor 506 . The A+N signal output to the vertical output line 404 is converted into digital data (A+N data) by AD conversion by the column circuit 405 and the storage unit 407 .

Until time tb6, the FD portion 503 holds the charges transferred from the photodiode 501a.

At time tb6, the vertical scanning circuit 403 sets the signals txa and txb to High level and then to Low level. As a result, the FD unit 503 stores, in addition to the charge of the photodiode 501a held up to time tb6, the charge accumulated by the photodiode 501b based on incident light, and the charge accumulated by the photodiode 501a during the period from time tb4 to time tb7. It retains the accumulated charge. Therefore, the voltage of the FD section 503 becomes a voltage corresponding to the sum of the charge of the photodiode 501 a and the charge of the photodiode 501 .

Amplification transistor 504 outputs the A+B+N signal to vertical output line 404 . The A+B+N signal output to the vertical output line 404 is converted into digital data (A+B+N data) by AD conversion by the column circuit 405 and storage unit 407 .

At time tb8, the vertical scanning circuit 403 changes the signal res to High level. Thereby, the voltage of the FD section 503 is reset.

Then, at time tb9, the vertical scanning circuit 403 changes the signal sel to Low level. This completes the selection of the pixels 401 in one row. After that, the vertical scanning circuit 403 sets the signal sel of the pixels 401 in the next readout row to High level.

The N data, A+N data, and A+B+N data held by the storage unit 407 are sequentially read out column by column from the storage unit 407 to the signal processing circuit 410 by the horizontal scanning circuit 409 . The signal processing circuit 410 outputs a difference signal (A data) between A+N data and N data and a difference signal between A+B+n data and N data (A+B data).

A signal processing unit (not shown) to which the data output from the imaging device is input obtains B data, which is the difference between the A data and the A+B data, and performs focus detection using the A data and the B data. Also, the signal processing unit generates an image using the A+B data.

Please refer to FIG. The imaging mode control unit 411 is a circuit block including the voltage control unit 104, the drive mode selection unit 105, the frequency control unit 106, and the signal generation unit 107 described in the first embodiment. Drive mode control unit 411 receives remaining life information from life information receiving unit 108 . Then, according to the length of remaining life indicated by the remaining life information, at least one of changing the voltage value supplied to all or part of the circuits in the image sensor 400 and changing the cycle of the control signal is performed. The circuits in this image sensor 400 are a pixel array 402, a vertical scanning circuit 403, a column circuit 405, a reference signal generating circuit 406, a storage section 407, a counter circuit 408, a horizontal scanning circuit 409, and a signal processing circuit 410. Alternatively, part of the operation of the pixel array 402, vertical scanning circuit 403, column circuit 405, reference signal generating circuit 406, storage unit 407, counter circuit 408, horizontal scanning circuit 409, and signal processing circuit 410 in the image sensor 400 is stopped. conduct.

As described in the formulas (1) to (4) of Embodiment 1, the lifetime of a semiconductor device depends on the current density J of the wiring, the wiring temperature T, the voltage Vg applied to the gate of the transistor, and the maximum substrate current Isub flowing through the semiconductor substrate. Limited by The imaging mode control unit 411 controls all or one of the current density J, the wiring temperature T, the voltage Vg applied to the gate of the transistor, and the maximum substrate current Isub flowing through the semiconductor substrate so that the remaining life of the imaging device 400 becomes long. Change the operating state so that the value of the part becomes smaller.

As a result, the processing amount of the signal processing of the image pickup apparatus is lowered, and the operation speed is lowered or the functions are partially stopped, but the image pickup operation can be continued and the life of the image pickup apparatus can be extended.

The life extension drive mode by the imaging mode control unit 411 may, for example, lower the frame rate. Specifically, the vertical scanning cycle of the vertical scanning circuit 403 is reduced compared to the normal drive mode. As the vertical scanning cycle decreases, the operation cycles of the reference signal generating circuit 406, the counter circuit 408, the horizontal scanning circuit 409, and the signal processing circuit 410 also decrease. Lowering the frame rate can be said to mean reducing the number of times the vertical scanning circuit 403 performs vertical scanning per unit time.

Further, as one of the life extension driving modes, signals may be read out from the pixels 401 in only a part of the pixel array 402 . That is, the number of pixels to be read is reduced in the life extension drive mode as compared with the normal drive mode. For example, the pixels 401 corresponding to the 4K2K resolution are read out in the normal drive mode, and the pixels 401 corresponding to the VGA resolution are read out in the life extension drive mode. In the column circuit 405 and the storage unit 407 as well, the horizontal scanning circuit 409 scans the storage unit 407 of only some columns according to the decrease in the number of pixels 401 to be read. This makes it possible to extend the remaining life of the imaging device while continuing the operation of the imaging device. Such an example is also included in the decrease in the processing amount of signal processing of the imaging apparatus.

As another example, the number of pixels for which the focus detection operation shown in FIG. 7B is performed may be set to be smaller in the life extension drive mode than in the normal drive mode. As a result, the number of AD conversions of the A+N signals in the column circuit 405 and the storage unit 407 can be reduced. This makes it possible to extend the remaining life of the imaging device while continuing the imaging operation. In particular, when it is important to extend the remaining life, the number of pixels for performing the focus detection operation in FIG. good. Such an example is also included in the decrease in the processing amount of signal processing of the imaging apparatus.

As described in the first embodiment, the life prediction unit 109 predicts the remaining life of the image pickup device 400 based on the latest operation state of the image pickup device 400 or history information related to the operation state. Obtain remaining life information.

Here, the operating state means a state when the image sensor 400 is in operation, such as operating voltage, operating frequency, etc., or a state resulting from operation such as temperature.

Acquisition of the remaining life information by the life prediction unit 109 is performed after the imaging operation is performed a plurality of times. The imaging operation referred to here refers to the operation of imaging a subject under various environments. In other words, rather than the imaging operation of the imaging device 400 under a known environment such as a test in the manufacturing factory of the imaging device, imaging under an environment in which the remaining life of the imaging device 400 changes depending on the environment in which the imaging device 400 is placed. refers to action. Therefore, the image pickup apparatus of this embodiment can predict the remaining life according to the environment in which the image pickup element 400 has actually been placed or placed, and the operation. Since this remaining life prediction is based on the environment and operation in which the imaging device 400 has actually been placed or placed, it is possible to predict the remaining life more accurately than prediction of life by simulation. .

According to the embodiment of the present invention described above, the imaging device 400 is driven to extend the life of the imaging device 400 based on the remaining life information. The problem is solved.

The imaging device of this embodiment can be used, for example, in a camera installed in an area where it is difficult to replace the imaging device. Examples of such cameras include cameras installed in remote locations (disaster monitoring (volcanic disasters, weather disasters), spacecraft, artificial satellites (meteorological observation satellites, exploration satellites, space stations, etc.), space exploration robots, etc.), There are surveillance cameras installed in conflict areas. Although it is not easy to replace the imaging device of these cameras even after the end of their life, it is desired that they continue to operate as imaging devices. Therefore, when the predicted remaining life falls below the first life threshold, the operation of the imaging device 400 is changed from the normal drive mode to the life extension drive mode. This makes it possible to extend the remaining operating life of the imaging device while continuing the imaging operation.

Further, the imaging device of the present embodiment can be used as an in-vehicle camera as an example of an imaging device mounted on a moving body. In the vehicle-mounted camera, it is not easy to replace the imaging element 400 while the vehicle is running even if the remaining life is less than the first life threshold. In such a case, it is useful to change the image pickup device 400 from the normal drive mode to the life extension drive mode to continue the image pickup operation and extend the life of the image pickup device 400 .

Further, acquisition of remaining life information by the life prediction unit 109 may be performed in parallel with the imaging operation. For example, in the case of a camera such as a camera installed in a remote location or a surveillance camera that continuously captures moving images, the remaining life prediction unit 109 may obtain remaining life information while the imaging operation is being performed. to This makes it possible to predict the remaining life of the imaging device 400 while continuing to shoot moving images. In other words, the life prediction unit 109 may acquire remaining life information while vertical scanning is being performed. Further, the life prediction unit 109 may acquire the remaining life information while the photoelectric conversion unit accumulates charges based on incident light. Further, the life prediction unit 109 may acquire the remaining life information while the column circuit 405 and the storage unit 407 are processing pixel signals (for example, AD conversion). In other words, the life prediction unit 109 may acquire the remaining life information while the imaging operation is performed a plurality of times.

As described above, the image pickup apparatus of the present embodiment has the effect of extending the life of the image pickup device while continuing the operation of the image pickup device even when the image pickup device is not easily replaced.

(Example 3)
The image pickup apparatus of this embodiment will be described with a focus on the differences from the second embodiment. The imaging apparatus of this embodiment can accurately obtain the remaining life information of the imaging element.

FIG. 8 is a diagram showing an imaging device 800 of this embodiment.

Unlike the imaging device 450 shown in FIG. 5, the imaging device 800 of this embodiment differs from the imaging device 450 in that it includes a state acquisition unit 812 and a life prediction unit 813 connected to the state acquisition unit 812 .

A case in which hot carrier injection (HC) is dominant as a factor of performance deterioration of the imaging device 400 will be described. In this case, the remaining life of the imaging device 400 is evaluated by the following equation (5), which is the ratio of the amount of change in drain current over time ΔId to the drain current Id of the transistor when the imaging device 400 starts operating as a new product. be.
Id _RATIO = ΔId/Id (5)

The state acquisition unit 812 measures the drain current Id of the transistor of the imaging device 400 . A measurement result is output to the life prediction unit 813 . The lifetime predicting unit 813 calculates the ratio of the known amount of change between the drain current Id of the MOS transistor at the beginning and the drain current Id of the latest MOS transistor.

Here, assume that the life time of the imaging element 400 is set to the time when the Id _RATIO is 0.1, that is, 10% in Equation (5). In this case, 10% is set as the threshold value of the Id _RATIO in the life prediction unit 813 . The life prediction unit 813 predicts the remaining life of the imaging device 400 by comparing the threshold value of 10% with the actual Id _RATIO .

As an example of a method for measuring the drain current Id of the transistor, a resistance element is provided in the state acquisition unit 812, one end of the resistance element is connected to the power supply voltage, and the other end of the resistance element is connected to the drain of the transistor of the imaging device 400. do. Then, the voltage values of both terminals of this resistance element are measured. As a result, the drain current can be measured using the formula V=IR (V is the voltage across the resistor element, R is the resistance value of the resistor element, and I is the current value flowing through the resistor element). By obtaining the time rate of change of the Id _RATIO , it is possible to predict the period until the Id _RATIO reaches 10%, that is, the remaining life.

Note that, as another method of acquiring the remaining life information, a change in the consumption current value consumed by the imaging device 400 may be measured. The drain current Id of the transistor can be measured indirectly by measuring the change in the consumption current value of the imaging device 400 when it is new and the consumption current value at the time of measurement. The remaining life can be predicted by obtaining the amount of change in the consumption current value over time.

(Example 4)
The image pickup apparatus of this embodiment will be described with a focus on the differences from the third embodiment. The configuration of the imaging device can be the same as in FIG.

In the image pickup apparatus of Example 3, the remaining life prediction was obtained from changes in the drain current Id. In this embodiment, another prediction method will be described.

As described in the first embodiment, semiconductor devices and imaging devices suffer performance degradation due to HC, EM, TDDB, and negative NBTI. This deterioration in performance determines the lifetime of the semiconductor device and the imaging device.

Life prediction regarding performance degradation due to HC is as described in Example 3.

Life time LT(EM) due to EM as a factor of performance deterioration is obtained by the formula (1) described in the first embodiment.

The life time LT(TDDB) due to TDDB as a performance degradation factor is obtained by the equation (2) described in the first embodiment.

The life time LT (NBTI) due to NBTI as a performance degradation factor can be calculated by the formula (4) described in the first embodiment.

Further, the life time LT(HC) due to HC as a performance deterioration factor is obtained by the equation (3) described in the first embodiment. It can also be obtained by the formula (5) described in the third embodiment.

The state acquisition unit 812 measures all or part of equations (1) to (5) to calculate the life of the imaging device 400 .

In order to calculate the lifetime of the imaging device 400 using equations (1) to (5), the state acquisition unit 812 obtains the current density J of the current flowing through the transistor or resistor provided in the imaging device 400, the wiring temperature T , measures the voltage value Vg applied to the gate of the transistor.

The method for obtaining the current density J can be the same as in the third embodiment.

As an example of a method for obtaining the wiring temperature T, the value of current flowing through a diode element (not shown) of the state obtaining unit 812 is measured. Since the value of the current flowing through the diode element generally depends on the temperature, the wiring temperature T can be obtained by measuring the value of the current.

The voltage value Vg can be the gate voltage applied to the transistor whose lifetime is to be predicted. As for the voltage value Vg, if the difference between the actual measurement value and the design value is expected to be substantially negligible, the design value may be used.

In this manner, the state acquisition unit 812 acquires each value used in formulas (1) to (5). State acquisition section 812 outputs each acquired value to life prediction section 813 . In other words, the state acquisition unit 812 acquires the temperature and current consumption of the imaging device 400 . Note that the state acquisition unit 812 may further acquire information on the cumulative operation time of the image sensor 400 and the frame rate.

The life prediction unit 813 obtains the life of the imaging device 400 using all or part of equations (1) to (5). When predicting the life using a plurality of formulas (1) to (5), the life prediction unit 813 sets the shortest life among the formulas used as the life of the imaging device 400 .

Thus, the life prediction unit 813 of this embodiment predicts the life based on the actual operating state of the imaging device 400 . This makes it possible to predict the service life with high precision.

The life of the imaging device 400 varies greatly depending on the usage environment. It changes depending on the temperature at which the imaging element 400 is placed, the number of shots, and the applied voltage. For example, an in-vehicle camera is exposed to a high temperature of 80° C. or more for a longer period of time in low latitude areas than in high latitude areas. In addition, high latitude regions are exposed to a low temperature of 0° C. or lower for a longer period of time than low latitude regions. Therefore, the actual life of the imaging device 400 may become shorter than the designed life. Therefore, by estimating the lifespan based on the actual operation of the image sensor 400, it is possible to prevent the image sensor 400 from suddenly stopping operation.

Although the state acquisition unit 812 and life prediction unit 813 are provided outside the imaging device 400 in this embodiment, the state acquisition unit 812 and life prediction unit 813 may be provided inside the imaging device 400 . In particular, when the state acquisition unit 812 is provided on the same semiconductor substrate as the image pickup device 400, it is placed under the same operating environment as the image pickup device 400, so life expectancy can be predicted with higher accuracy.

(Example 5)
The image pickup apparatus of this embodiment will be described with a focus on the differences from the fourth embodiment.

FIG. 9 is a diagram showing the configuration of the imaging apparatus of this embodiment. The imaging apparatus of this embodiment further has a memory unit 920 in contrast to the imaging apparatus of the fourth embodiment.

The memory unit 920 holds history information related to operating states.

The life prediction unit 913 of this embodiment uses the history information held by the memory unit 920 and the information output from the state acquisition unit 812 to predict the remaining life and generate the remaining life information. The history information is, for example, the accumulated operating time of the imaging device and the history of the imaging mode of the imaging device. The imaging mode includes the cycle (frame rate) of outputting an image signal from the imaging element 400, the image size, whether or not to execute the focus detection function, and the like.

As an example of the operation, the remaining life predicted by the life prediction unit 913 using all or part of the equations (1) to (5) as described in the fourth embodiment is stored in the history information held by the memory unit 920. corrected using

As another example of operation, the memory unit 920 retains the Id _RATIO obtained using Equation (5) each time the remaining life prediction is performed. Then, the life prediction unit 913 obtains the temporal change of the Id _RATIO from the Id _RATIO at each time held in the memory unit 920 . Based on this change in Id _RATIO over time, prediction of the time when the Id _RATIO becomes 10% or more (that is, remaining life prediction) is performed. As a result, more accurate remaining life prediction can be performed.

Moreover, the value stored in the memory unit 920 is not limited to the drain current Id of the transistor, and may be the values of all or part of equations (1) to (5).

Note that the memory unit 920 may be provided outside the imaging device 800 . Also, it is desirable that the memory unit 920 is a non-volatile memory. When the memory unit 920 is provided outside the imaging device 800, the manufacturing processes of the imaging device 800 and the memory unit 920 can be separated. As a result, restrictions on the manufacturing processes of the imaging device 800 and the memory unit 920 can be reduced.

As described above, the imaging apparatus of this embodiment includes the memory unit 920 that holds history information related to the operating state. By performing remaining life prediction using the history information held by the memory unit 920, the remaining life can be predicted with high accuracy.

Note that the imaging apparatus described in the embodiments so far can be a stacked sensor in which a plurality of chips are stacked. For example, a pixel array 402 is arranged on a first chip, and a column circuit 405, a reference signal generation circuit 406, a storage section 407, a counter circuit 408, a horizontal scanning circuit 409, a signal processing circuit 410, and an imaging mode control section 411 are arranged on a second chip. may be provided. In this case, the second chip may further include a lifetime predictor 109 . Further, a third chip provided with a signal processing section for generating an image may be further stacked. In this case, the life prediction unit 109 may be provided in the third chip.

(Example 6)
FIG. 10 is a block diagram showing the configuration of an imaging system 500 according to this embodiment. An imaging system 500 of this embodiment includes an imaging device 200 to which any configuration of the imaging devices described in the above embodiments is applied. Specific examples of the imaging system 500 include a digital still camera, a digital camcorder, a surveillance camera, and the like. FIG. 10 shows a configuration example of a digital still camera to which any one of the imaging apparatuses of the above embodiments is applied as an imaging apparatus 200. As shown in FIG.

An imaging system 500 illustrated in FIG. 10 includes an imaging device 200, a lens 5020 for forming an optical image of a subject on the imaging device 200, an aperture 504 for varying the amount of light passing through the lens 5020, and a lens for protecting the lens 5020. has a barrier 506 of A lens 5020 and a diaphragm 504 are an optical system that condenses light on the imaging device 200 .

The imaging system 500 also has a signal processing section 5080 that processes an output signal output from the imaging device 200 . The signal processing unit 5080 performs a signal processing operation of performing various corrections and compressions on an input signal and outputting the signal as necessary. The signal processing unit 5080 may have a function of performing AD conversion processing on the output signal output from the imaging device 200 . In this case, the imaging device 200 does not necessarily have an AD conversion circuit inside.

The imaging system 500 further includes a buffer memory section 510 for temporarily storing image data, and an external interface section (external I/F section) 512 for communicating with an external computer or the like. Further, the imaging system 500 includes a recording medium 514 such as a semiconductor memory for recording or reading imaging data, and a recording medium control interface section (recording medium control I/F section) 516 for recording or reading the recording medium 514. have Note that the recording medium 514 may be built in the imaging system 500 or may be detachable.

The imaging system 500 further includes an overall control/calculation unit 518 that performs various calculations and controls the entire digital still camera, and a timing generation unit 520 that outputs various timing signals to the imaging device 200 and the signal processing unit 5080 . Here, the timing signal and the like may be input from the outside, and the imaging system 500 only needs to have at least the imaging device 200 and the signal processing section 5080 that processes the output signal output from the imaging device 200 . The overall control/calculation unit 518 and the timing generation unit 520 may be configured to implement some or all of the control functions of the imaging device 200 .

The imaging device 200 outputs the image signal to the signal processing section 5080 . The signal processing unit 5080 performs predetermined signal processing on the image signal output from the imaging device 200 and outputs image data. Also, the signal processing unit 5080 generates an image using the image signal.

By configuring an imaging system using the imaging apparatus according to each of the embodiments described above, it is possible to realize an imaging system capable of acquiring a higher-quality image.

(Example 7)
An imaging system and a moving object according to this embodiment will be described with reference to FIGS. 11 and 12. FIG.

FIG. 11 is a schematic diagram showing a configuration example of an imaging system and a moving object according to this embodiment. FIG. 12 is a flowchart showing the operation of the imaging system according to this embodiment.

In this embodiment, an example of an imaging system for an in-vehicle camera is shown. FIG. 11 shows an example of a vehicle system and an imaging system mounted thereon. The imaging system 701 includes an imaging device 702 , an image preprocessor 715 , an integrated circuit 703 and an optical system 714 . An optical system 714 forms an optical image of a subject on the imaging device 702 . The imaging device 702 converts the optical image of the object formed by the optical system 714 into an electrical signal. The imaging device 702 is one of the imaging devices of the embodiments described above. An image preprocessing unit 715 performs predetermined signal processing on the signal output from the imaging device 702 . The functionality of the image preprocessor 715 may be incorporated within the imaging device 702 . The imaging system 701 is provided with at least two sets of an optical system 714 , an imaging device 702 , and an image preprocessing unit 715 . It's becoming

The integrated circuit 703 is an integrated circuit for use in an imaging system, and includes an image processing unit 704 including a memory 705 , an optical distance measurement unit 706 , a parallax calculation unit 707 , an object recognition unit 708 and an abnormality detection unit 709 . An image processing unit 704 performs image processing such as development processing and defect correction on the output signal of the image preprocessing unit 715 . A memory 705 temporarily stores captured images and stores defect positions of captured pixels. An optical distance measurement unit 706 performs focusing and distance measurement on a subject. A parallax calculation unit 707 calculates parallax (phase difference of parallax images) from a plurality of image data acquired by a plurality of imaging devices 702 . An object recognition unit 708 recognizes subjects such as cars, roads, signs, and people. The abnormality detection unit 709 notifies the main control unit 713 of the abnormality when detecting the abnormality of the imaging device 702 .

The integrated circuit 703 may be implemented by specially designed hardware, software modules, or a combination thereof. Moreover, it may be implemented by FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or the like, or by a combination thereof.

A main control unit 713 integrates and controls the operations of the imaging system 701, the vehicle sensor 710, the control unit 720, and the like. A method in which the imaging system 701, the vehicle sensor 710, and the control unit 720 have separate communication interfaces without having the main control unit 713, and each of them transmits and receives control signals via a communication network (for example, CAN standard). can also take

The integrated circuit 703 has a function of receiving a control signal from the main control unit 713 or transmitting a control signal and setting values to the imaging device 702 by its own control unit. For example, the integrated circuit 703 transmits settings for pulse-driving the voltage switch 13 in the imaging device 702, settings for switching the voltage switch 13 for each frame, and the like.

The imaging system 701 is connected to a vehicle sensor 710, and can detect the running state of the own vehicle such as vehicle speed, yaw rate, and steering angle, the environment outside the own vehicle, and the states of other vehicles and obstacles. The vehicle sensor 710 also serves as distance information acquisition means for acquiring distance information from the parallax image to the object. The imaging system 701 is also connected to a driving support control unit 711 that performs various driving support functions such as automatic steering, automatic cruise, and anti-collision functions. In particular, regarding the collision determination function, based on the detection results of the imaging system 701 and the vehicle sensor 710, it is possible to estimate a collision with another vehicle/obstacle and determine whether or not there is a collision. As a result, avoidance control when a collision is presumed and safety device activation at the time of collision are performed.

The imaging system 701 is also connected to an alarm device 712 that issues an alarm to the driver based on the determination result of the collision determination section. For example, if the collision determination unit determines that there is a high probability of collision, the main control unit 713 controls the vehicle to avoid collisions and reduce damage by applying the brakes, releasing the accelerator, or suppressing engine output. conduct. The alarm device 712 warns the user by sounding an alarm such as sound, displaying alarm information on a display unit screen of a car navigation system or a meter panel, or vibrating a seat belt or steering wheel.

In this embodiment, the image pickup system 701 photographs the surroundings of the vehicle, for example, the front or rear. FIG. 11B shows an arrangement example of the image pickup system 701 when the image pickup system 701 picks up an image in front of the vehicle.

Two imaging devices 702 are arranged in front of the vehicle 700 . Specifically, if the center line of the vehicle 700 with respect to the forward/retreat direction or the outer shape (for example, the width of the vehicle) is regarded as the axis of symmetry, and the two imaging devices 702 are arranged symmetrically with respect to the axis of symmetry, the vehicle 700 and the subject are arranged. This is preferable for obtaining information on the distance to the photographed object and determining the possibility of collision. In addition, the imaging device 702 is preferably arranged so as not to obstruct the driver's field of view when the driver visually recognizes the situation outside the vehicle 700 from the driver's seat. It is preferable that the warning device 712 be arranged so as to be easily visible to the driver.

Next, failure detection operation of the imaging device 702 in the imaging system 701 will be described with reference to FIG. The failure detection operation of the imaging device 702 is performed according to steps S810 to S880 shown in FIG.

Step S<b>810 is a step of performing settings for startup of the imaging device 702 . That is, the setting for the operation of the imaging device 702 is transmitted from the outside of the imaging system 701 (for example, the main control unit 713) or the inside of the imaging system 701, and the imaging operation and failure detection operation of the imaging device 702 are started.

Next, in step S820, pixel signals are obtained from effective pixels. Also, in step S830, an output value is obtained from a failure detection pixel provided for failure detection. This failure detection pixel has a photoelectric conversion section like the effective pixel. A predetermined voltage is written in the photoelectric conversion unit. The failure detection pixel outputs a signal corresponding to the voltage written to the photoelectric conversion section. Note that steps S820 and S830 may be reversed.

Next, in step S840, whether or not the expected output value of the failure-detected pixel corresponds to the actual output value from the failure-detected pixel is determined.

As a result of the pertinence determination in step S840, if the expected output value and the actual output value match, the process proceeds to step S850, it is determined that the imaging operation is performed normally, and the processing step proceeds to step S860. and migrate. In step S860, the pixel signals of the scanning line are transmitted to the memory 705 for temporary storage. After that, the process returns to step S820 to continue the failure detection operation.

On the other hand, if the result of the pertinence determination in step S840 is that the expected output value and the actual output value do not match, the process proceeds to step S870. In step S870, it is determined that there is an abnormality in the imaging operation, and an alarm is issued to the main control unit 713 or the alarm device 712. FIG. The alarm device 712 causes the display unit to display that an abnormality has been detected. After that, in step S880, the imaging device 702 is stopped, and the operation of the imaging system 701 ends.

In this embodiment, the flowchart is looped for each line, but the flowchart may be looped for a plurality of lines, or the failure detection operation may be performed for each frame.

Note that the issuing of the warning in step S870 may be notified to the outside of the vehicle via a wireless network.

In addition, in this embodiment, control for avoiding collision with other vehicles has been described, but it is also applicable to control for automatically driving following another vehicle or control for automatically driving so as not to stray from the lane. . Furthermore, the imaging system 701 can be applied not only to vehicles such as the own vehicle, but also to moving objects (moving devices) such as ships, aircraft, and industrial robots. In addition, the present invention can be applied not only to mobile objects but also to devices that widely use object recognition, such as intelligent transportation systems (ITS).

[Modified embodiment]
The present invention is not limited to the above embodiments, and various modifications are possible.

For example, an example in which a part of the configuration of any one embodiment is added to another embodiment, or an example in which a part of the configuration of another embodiment is replaced is also an embodiment of the present invention.

Moreover, the above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed to be limited by these exemplifications. That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

100 semiconductor device 101 to 103 circuit block 105 drive mode selection unit 109, 813, 913 life prediction unit 150 semiconductor device 400 image pickup device 402 pixel array 411 image pickup mode control unit 450 image pickup device 812 state acquisition unit 920 memory unit

Claims (20)

A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition ;
when the drive control unit drives the semiconductor element under the first condition, the lifespan obtaining unit obtains the remaining lifespan information at a first interval;
When the drive control unit drives the semiconductor device under the second condition, the life acquisition unit acquires the remaining life information at a second interval shorter than the first interval. semiconductor device. A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
A semiconductor device, wherein the remaining life information is acquired using the following equation (1).
LT=A×J− ⁿ ×exp (Ea/kT) (1)
LT: remaining life information, A: coefficient, Ea: activation energy, J: current density of wiring, n: current acceleration factor, k: Boltzmann constant, T: temperature 2. The semiconductor device according to claim 1, wherein the remaining life information is acquired using the following equation (1) .
LT=A×J− ⁿ ×exp (Ea/kT) (1)
LT: remaining life information, A: coefficient, Ea: activation energy, J: current density of wiring, n: current acceleration factor, k: Boltzmann constant, T: temperature A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
A semiconductor device, wherein the remaining life information is acquired using the following equation (2).
LT=A×exp(−γVg×Vg)×exp(Ea/kT) (2)
LT: remaining life information, Vg: voltage applied to gate of transistor, γVg: voltage acceleration factor 3. The semiconductor device according to claim 1, wherein the remaining life information is acquired using the following equation (2) .
LT=A×exp(−γVg×Vg)×exp(Ea/kT) (2)
LT: remaining life information, Vg: voltage applied to the gate of the transistor, γVg: voltage acceleration factor A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
A semiconductor device, wherein the remaining life information is acquired using the following equation (3).
LT=A×Isub− ^m ×exp (Ea/kT) (3)
LT: remaining life information, Isub: current flowing in the semiconductor substrate, m: substrate current dependent coefficient, Ea: activation energy, k: Boltzmann constant, T: temperature 3. The semiconductor device according to claim 1, wherein the remaining life information is acquired using the following equation (3) .
LT=A×Isub− ^m ×exp (Ea/kT) (3)
LT: remaining life information, Isub: current flowing in the semiconductor substrate, m: substrate current dependent coefficient, Ea: activation energy, k: Boltzmann constant, T: temperature A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
A semiconductor device, wherein the remaining life information is acquired using the following equation (4).
LT=A× ^Vgn ×exp(Ea/kT) (4)
LT: remaining life information, A: coefficient, Ea: activation energy, k: Boltzmann constant, T: temperature, n: current acceleration factor 3. The semiconductor device according to claim 1, wherein the remaining life information is acquired using the following equation (4) .
LT=A×Vgn ^× exp(Ea/kT) (4)
LT: remaining life information, A: coefficient, Ea: activation energy, k: Boltzmann constant, T: temperature, n: current acceleration factor the semiconductor device comprises a transistor;
10. The semiconductor device according to claim 1, wherein said lifespan obtaining unit obtains said remaining lifespan information based on a temporal change in drain current flowing through said transistor. A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a time acquisition unit that acquires time information indicating an operation elapsed time of the semiconductor device,
when the time information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the time information indicates a second length longer than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition ;
when the drive control unit drives the semiconductor element under the first condition, the time acquisition unit acquires the time information at a first interval;
When the drive control unit drives the semiconductor element under the second condition, the time acquisition unit acquires the time information at a second interval shorter than the first interval. semiconductor device. the first condition is a condition that the operating frequency of the semiconductor element is the first frequency;
12. The semiconductor according to any one of claims 1 to 11 , wherein the second condition is a condition that the operating frequency of the semiconductor element is a second frequency lower than the first frequency. Device. the first condition is a condition that a power supply voltage supplied to the semiconductor element is a first voltage;
13. The method according to claim 1 , wherein the second condition is a condition that a power supply voltage supplied to the semiconductor element is a second voltage having a smaller absolute value than the first voltage. 10. The semiconductor device according to claim 1. the first condition is a condition that the operating frequency of the semiconductor element is the first frequency;
14. The semiconductor according to any one of claims 1 to 13 , wherein the second condition is a condition that the operating frequency of the semiconductor element is a second frequency lower than the first frequency. Device. 15. The semiconductor device according to any one of claims 1 to 14, wherein the semiconductor element is arranged over a plurality of rows and a plurality of columns, and has a plurality of pixels each having a photoelectric conversion portion. The semiconductor element includes a scanning circuit that performs vertical scanning for scanning the plurality of pixels row by row,
the first condition is a condition for performing the vertical scanning a first number of times per unit time;
16. The semiconductor device according to claim 15 , wherein said second condition is a condition that said vertical scanning is performed a second number of times less than said first number of times per unit time. A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a life acquisition unit that acquires remaining life information indicating the remaining life of the semiconductor device,
when the remaining life information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the remaining life information indicates a second length that is shorter than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
The semiconductor element comprises a plurality of pixels arranged over a plurality of rows and a plurality of columns, each having a photoelectric conversion unit,
The semiconductor element includes a scanning circuit that performs vertical scanning for scanning the plurality of pixels row by row,
the first condition is a condition for performing the vertical scanning a first number of times per unit time;
The semiconductor device, wherein the second condition is a condition that the vertical scanning is performed a second number of times less than the first number of times per unit time . A semiconductor device that performs signal processing, a drive control unit that controls driving of the semiconductor device, and a time acquisition unit that acquires time information indicating an operation elapsed time of the semiconductor device,
when the time information indicates a first length, the drive control unit drives the semiconductor element under a first condition;
When the time information indicates a second length longer than the first length, the drive control unit reduces the processing amount of the signal processing compared to when the semiconductor element is driven under the first condition. driving the semiconductor element under a second condition in which the remaining life of the semiconductor element is longer than when the semiconductor element is driven under the first condition;
The semiconductor element comprises a plurality of pixels arranged over a plurality of rows and a plurality of columns, each having a photoelectric conversion unit,
The semiconductor element includes a scanning circuit that performs vertical scanning for scanning the plurality of pixels row by row,
the first condition is a condition for performing the vertical scanning a first number of times per unit time;
The semiconductor device, wherein the second condition is a condition that the vertical scanning is performed a second number of times less than the first number of times per unit time . 19. An imaging system, comprising: the semiconductor device according to claim 15; and a signal processing section that processes a signal output from the semiconductor element. A moving body having the semiconductor device according to any one of claims 15 to 18 ,
A moving object, further comprising a control unit for controlling movement of the moving object.

Images