JP7257299B2 - Imaging system and imaging system application equipment - Google Patents

Description

The present invention relates to an imaging system using an image sensor and imaging system application equipment.

2. Description of the Related Art Imaging systems having image sensors are installed in nuclear power plants and radiation utilization facilities in order to monitor the inside of the plants and facilities. In addition, an imaging system having an image sensor is utilized for internal investigation for decommissioning of a nuclear power plant. Image sensors included in these imaging systems include imaging tubes, which are vacuum tubes, and solid-state imaging devices using semiconductors.

As described in Patent Document 1, in a solid-state image sensor, a charge photoelectrically converted by a light-receiving element in a pixel is normally transferred to a CMOS amplifier (complementary metal-insulator-semiconductor field effect transistor). , CMOS-FET). The amplified voltage signal is sequentially scanned by a vertical scanning circuit and a horizontal scanning circuit to obtain a voltage signal proportional to the amount of light of each image, that is, the luminance. Thus, an image of the object (subject) is acquired.

However, CMOS amplifiers are known to be susceptible to deterioration due to exposure to radiation. One reason for this is as follows. When a MOSFET is irradiated with radiation, electrons and holes are generated in the gate oxide film (insulating film). Since electrons in the oxide film have higher mobility than holes, they escape from the oxide film, but holes have lower mobility and accumulate in the oxide film as positive charges. This is because the charge accumulated in the oxide film causes the threshold of the MOSFET to fluctuate, causing adverse effects such as a failure to stably amplify the signal.

In general, circuit elements that are used in harsh environments such as radiation environments are shielded with lead or the like, or are kept away from radiation sources. However, image sensors, cameras, and imaging devices cannot be shielded by lead plates that do not transmit light.

JP 2014-39159 A

As described above, a conventional solid-state image sensor type image sensor normally uses a CMOS amplifier, and the CMOS amplifier has a problem that its characteristics are likely to deteriorate due to irradiation with radiation.

Attempts have been made to use vacuum tubes such as camera tubes to address this problem, but camera tubes are physically large and heavy due to the principle of spatially scanning electron beams. There is a problem.

On the other hand, in application equipment such as robot systems, there is a demand to mount a plurality of image sensors, and a compact and lightweight image sensor is required.

A problem to be solved by the present invention is to provide an image pickup system and an image pickup system application equipment using a small and light image sensor which is excellent in environmental performance such as radiation resistance.

A representative example of the invention disclosed in the present application is as follows. That is, the imaging system includes a sensor section and an image processing section, wherein the sensor section includes a transmission/reception section and a sensor element, the sensor element includes a light receiving element and a charging transistor for each pixel, and the sensor The element has one or a plurality of threshold determination circuits for determining whether the terminal voltage of the light receiving element is equal to or greater than a predetermined threshold, and the sensor element changes the exposure time of the light receiving element for each sub-image to determine the luminance. A plurality of sub-images corresponding to different luminance thresholds are acquired by changing a threshold value, and the image processing section processes the signals of the plurality of sub-images to obtain a gradation image. Other solutions are described later in the detailed description.

According to one aspect of the present invention, it is possible to provide an imaging device and an imaging system that have excellent environmental resistance and can be used even in a harsh environment. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 is a schematic diagram showing the configuration of an imaging system of a first embodiment; FIG. FIG. 2 is a schematic diagram showing the configuration of the sensor element of the first embodiment; FIG. 3 is a schematic diagram showing the configuration of a pixel of the first embodiment; FIG. 1 is a circuit diagram showing a threshold value determination circuit of a first embodiment; FIG. 5 is a circuit diagram showing another example of the threshold value determination circuit of the first embodiment; FIG. FIG. 2 is a schematic diagram showing the configuration of a pixel of a conventional sensor element; FIG. 4 is a timing chart schematically showing a readout sequence from pixels in the sensor element of the first embodiment; FIG. 4 is a schematic diagram showing a method of forming a gradation image according to the first embodiment; 4 is a logic table showing the process of constructing a gradation image of the first embodiment; FIG. 10 is a circuit diagram of a circuit that implements the process of forming the gradation image shown in FIG. 9; FIG. 4 is a schematic diagram showing a state in which image data of sub-images of the first embodiment are stored in a memory; FIG. 12 is a truth table for realizing the processing for constructing the gradation image shown in FIG. 11; FIG. FIG. 10 is a diagram showing the configuration of the sensor element of the second embodiment; FIG. 11 is a timing diagram schematically showing a readout sequence from pixels in the sensor element of the second embodiment; FIG. 11 is a schematic diagram showing the relationship between the amount of light incident on the light receiving element of the third embodiment and the terminal voltage; FIG. 12 is a schematic diagram showing the relationship between the readout sequence from each pixel and the charging voltage Vb0 in the third embodiment; FIG. 14 is a schematic diagram showing the relationship between the readout sequence from each pixel and the charging voltage Vb0 in the fourth embodiment; FIG. 12 is a diagram showing a robot system as an example of applied equipment of the seventh embodiment; FIG. 14 is a schematic diagram showing the configuration of a pixel in an eighth embodiment; FIG. 11 is a schematic diagram showing the configuration of a sensor element of an eighth embodiment;

<Example 1>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments, and for example, a plurality of embodiments can be combined or arbitrarily modified without departing from the technical idea of the present invention.

Moreover, in this specification, the same reference numerals are given to the same members, and overlapping descriptions are omitted. For convenience of illustration, the contents of the drawings may be changed from the actual configuration within the scope that does not impair the gist of the present invention.

FIG. 1 is a schematic diagram showing the configuration of an imaging system 1 of this embodiment.

The imaging system 1 of this embodiment is composed of a sensor section 10 and an image processing section 20 . As illustrated, the sensor unit 10 and the image processing unit 20 may be configured separately and connected by the cable 30 or wirelessly, but the sensor unit 10 and the image processing unit 20 may be integrated.

The sensor unit 10 includes a sensor element 13 that detects an external image, a sensor control unit 11 that controls processing by circuits in the sensor element 13, and a signal transmission/reception unit that transmits and receives signals between the sensor unit 10 and the image processing unit 20. a portion 12; The sensor control unit 11 and the transmission/reception unit 12 are functionally separated, and the sensor element 13, the sensor control unit 11, and the transmission/reception unit 12 may be provided in one IC chip.

(Structure of sensor element 13)
FIG. 2 is a schematic diagram showing the structure of the sensor element 13. As shown in FIG.

In FIG. 2, four pixels 130 are shown for clarity of explanation. The actual sensor element 13 has a larger number of pixels, typically 100 to 2000 rows and 100 to 4000 columns.

Two horizontal wirings and one vertical wiring are connected to each pixel.

One of the horizontal wirings is a charge scanning line (Charge scanning line) 135 for charging, and has a function of charging the light receiving element 131 of the pixel 130 as described later. Another horizontal wiring is a read scanning line (Read scanning line) 136 that designates the signal read timing. The charge scanning line 135 is connected to the charge address terminal (“CA” in the drawing) of each pixel 130 . The readout scanning line 136 is connected to the readout address terminal (“RA” in the drawing) of each pixel 130 . The charging scanning lines 135 and the reading scanning lines 136 are provided by the number of rows of each pixel 130 .

The charging scanning line 135 is connected to a charging scanning circuit (Charge scanning circuit) 138 . The read scanning line 136 is connected to a read scanning circuit 139 (Read scanning circuit). The charging scanning circuit 138 and the reading scanning circuit 139 respectively output appropriate voltage waveforms to scan each pixel 130 . This scanning method will be described in detail later.

The vertical wiring is the data line 137 . A data line 137 is wired to each pixel 130 .

The data line 137 is connected to the data output terminal (“D” in the drawing) of each pixel 130 . The data line 137 has a function of reading the signal detected by the light receiving element 131 in the pixel 130 . The data line 137 is connected to the horizontal scanning circuit 140 , sequentially reads out signals detected by the light receiving element 131 in each pixel 130 , and transfers them to the transmission/reception section 12 .

Although not shown, the pixels 130 may be provided with color filters. For example, a Bayer filter in which three colors of red (R), green (G), and blue (B) are arranged in an RGGB pattern can be used.

(Pixel configuration)
FIG. 3 is a schematic diagram showing the configuration of each pixel 130. As shown in FIG.

Each pixel 130 has a light receiving element 131 , a storage capacitor 132 , a charge transistor (Charge transistor) 133 and a threshold determination circuit 134 .

In the drawings representing the circuit diagrams described herein, downward hollow arrows indicate connections to ground potential. Here, the ground potential means the reference potential in the sensor element 13, and does not have to coincide with the ground potential of the signal processing section. If necessary, the reference potential (ground potential) in the sensor element 13 may be biased with respect to the ground potential of the signal processing section.

In this embodiment, it is preferable to use a silicon photodiode as the light receiving element 131 .

The storage capacitor 132 may be provided independently of the light receiving element 131, or the junction capacitance (parasitic capacitance) of the light receiving element 131 may be used.

When a photodiode is used as the light receiving element 131, the equivalent circuit of the junction capacitance of the light receiving element 131 is the same as the holding capacitor 132 in FIG. Therefore, when the junction capacitance of the light receiving element 131 has a sufficient capacity as the holding capacitance 132, instead of providing the holding capacitance 132 separately from the light receiving element 131, the junction capacitance can be used to function as the holding capacitance 132. good. In this specification, the holding capacitor 132 includes the junction capacitance (parasitic capacitance) of the light receiving element 131 as described above.

That is, in this specification, the storage capacitance 132 also includes the junction capacitance of the light receiving element 131 . The effects of the present invention can also be obtained when the junction capacitance of the light receiving element 131 is substituted for the storage capacitance 132 .

A method of detecting external light in each pixel 130 will be described with reference to FIG. When an appropriate voltage is applied to the charging scanning line 135 to turn on the charging transistor 133 (ie, conductive state), the light receiving element 131 and the holding capacitor 132 are charged to the charging voltage Vb0. Note that the charging transistor 133 may be a bipolar transistor or a MOSFET.

In this state, when the light receiving element 131 is not irradiated with light, the terminal voltage of the holding capacitor 132 is held. On the other hand, when the light-receiving element 131 is irradiated with light, the light-receiving element 131 is in a conducting state, so that the charge of the holding capacitor 132 leaks out little by little, and the terminal voltage of the holding capacitor 132 decreases.

(Threshold determination circuit 134)
A terminal of the holding capacitor 132 is connected to an input terminal IN of the threshold determination circuit 134 .

The threshold determination circuit 134 outputs a signal 1 from the output terminal O when the terminal voltage of the holding capacitor 132 is lower than a preset voltage threshold Vth, and outputs a signal 0 from the output terminal O when it is higher than the voltage threshold Vth. Output. That is, when the luminance of an external subject exceeds a predetermined luminance threshold value Bth, the voltage of the storage capacitor 132 of the pixel 130 at the corresponding position is lowered due to the leakage current of the light receiving element 131 and becomes smaller than Vth. Circuit 134 outputs a 1 as an output signal. On the other hand, when the brightness of the object is lower than the brightness threshold value Bth, the voltage of the storage capacitor 132 of the pixel 130 at the corresponding position is higher than Vth, so the threshold determination circuit 134 outputs 0 as an output signal.

Note that the output signal 1 means the signal state 1 of the logic circuit. For example, a high voltage state (high level) is a signal "1", and a low voltage state (low level) is a signal "0". Conversely, the Low level may be signal "1" and the High level may be signal "0". Alternatively, a current level or the like may be used instead of the voltage level. In this embodiment, the signal "1" corresponds to the high level of the voltage.

In this way, a signal "1" is output from the pixels 130 corresponding to the locations of the subject luminance threshold Bth or higher, and a signal "0" is output from the pixels 130 corresponding to the luminance threshold Bth or lower. That is, a binary image based on the brightness threshold Bth can be acquired.

Since a binary signal flows through each data line 137, the switch 141 may be a logic circuit.

(Output impedance of threshold determination circuit 134)
The threshold decision circuit 134 has a read enable terminal RE (Read Enable). A read enable terminal RE is connected to the read scan line 136 . A signal "0" or "1" is output from the output terminal O in accordance with the operation described above during a period in which the enable signal is input to the read enable terminal RE (read enable period).

During a period (read disable period) in which no enable signal is input to the read enable terminal RE, the output terminal O transitions to a higher impedance state. Thus, by setting the output impedance of the output terminal O to a high impedance during the invalid readout period, even if a plurality of pixels 130 are connected to the data line 137, the output of the threshold determination circuit 134 of the selected pixel 130 is signal can be extracted.

(Example of implementation of threshold determination circuit 134)
FIG. 4 is a circuit diagram showing the threshold determination circuit 134 of this embodiment.

The threshold determination circuit 134 of this embodiment is composed of a transistor 1 (Tr1) that is a PNP transistor, a transistor 2 (Tr2) that is an NPN transistor, and a resistor (R). An input terminal of the threshold determination circuit 134 is connected to the base of the transistor 2 (Tr2). When the input terminal of the threshold determination circuit 134 is higher than the threshold voltage Vth of the transistor 2 (Tr2), the collector of the transistor 2 (Tr2) is maintained (latched) at the power supply voltage Vcc. The collector of Tr2 becomes the output terminal of the threshold decision circuit 134 .

The threshold decision circuit 134 used in this embodiment employs a latch circuit, and the output signal maintains the High level or the Low level while the power supply voltage Vcc is applied. Thus, by using the latch circuit for the threshold value determination circuit 134, there is an effect that malfunction can be reduced.

When the voltage of the RE terminal in FIG. 4 is set to Low level, the two transistors Tr1 and Tr2 are rendered non-conductive, so that the output impedance of the output terminal O becomes a high impedance state.

FIG. 5 is a circuit diagram showing another example of the threshold determination circuit 134. As shown in FIG.

In the threshold determination circuit 134 shown in FIG. 5, the operations of the two transistors Tr1 and Tr2 are the same as in FIG. 4, with the addition of the third transistor, the readout transistor Tr3. The read transistor Tr3 is turned ON only during the period in which the read enable pulse is applied, and a signal voltage is output to the data line 137. FIG. Since the readout transistor Tr3 is in the OFF state during the readout invalid period, the output impedance of the output terminal O of the threshold determination circuit 134 becomes higher than that of the circuit in FIG.

Increasing the output impedance of the threshold determination circuit 134 of the unselected pixels 130 is more desirable when the number of scanning lines is large. The reason for this is that each data line 137 is connected to the threshold determination circuit 134 for the number of scanning lines. This is because the impedance due to the threshold decision circuit 134 of the non-selected pixels can be maintained at a sufficiently high level.

The circuits shown in FIGS. 4 and 5 are examples of the threshold determination circuit 134, and the effects of the present invention can be obtained by using other circuits as long as they satisfy the characteristics of the threshold determination circuit 134 described above. Needless to say.

(Comparison with conventional example)
Next, a comparison with the conventional example will be made. FIG. 6 is a schematic diagram showing the configuration of a pixel 130 of a conventional sensor element 13. As shown in FIG.

FIG. 6 shows an example of a CMOS sensor (complementary MOS field effect transistor type sensor), which is a mainstream system as an imaging device for consumer cameras.

In the prior art shown in FIG. 6, the voltage of the holding capacitor 132 is amplified by the pixel amplifier, and the voltage value V2 proportional to the charge accumulated in the holding capacitor 132 is output as an analog signal. A CMOS type FET (field effect transistor) is used for the pixel amplifier. FET2 and FET3 in FIG. 6 constitute a pixel amplifier. In this pixel amplifier, the FET2 operates as a source follower circuit, and performs charge amplification by converting the charge applied to the gate of the FET2 into a low-impedance voltage signal.

When an appropriate voltage is applied to the read scanning line 136 to turn on the FET 3 , an analog signal voltage, which is the output signal of the pixel amplifier, is output to the data line 137 and sequentially output by the horizontal scanning circuit 140 .

In the prior art, since a voltage signal corresponding to the luminance level of the object is output in analog form, an image with gradation can be obtained.

However, since the CMOS pixel amplifier used in the circuit within the pixel in FIG. 6 has low resistance to radiation exposure, there is a problem that the sensor element 13 is likely to deteriorate when used in a harsh environment such as the presence of radiation. . On the other hand, the present invention uses an element having a higher radiation resistance than the CMOS amplifier, and therefore has the effect of suppressing deterioration of characteristics under a radiation environment.

In particular, as shown in FIG. 4, which is the present embodiment of the present invention, bipolar transistors are known to have higher radiation resistance than CMOS FETs, and it is more preferable to use bipolar transistors.

Moreover, in the conventional technology described above, the changeover switch 141 and the transmission/reception circuit 142 must handle analog signals. On the other hand, in the present invention, the signals processed by the changeover switch 141 and the transmission/reception circuit 142 are binary signals, so the circuit configuration is simpler than that of the prior art.

(Gradation of binary image)
As described above, the image obtained by the pixels 130 of this embodiment is a binary image obtained by binarizing the subject with a certain luminance threshold.

A method for constructing a gradation image from a binary image will be described below. In this embodiment, a binary image is acquired with a plurality of brightness thresholds Bth[n], and each of them is called a sub-image SIm[n] (Sub-Image). Here, the suffix n is a number for distinguishing between the luminance threshold and the sub-image, n=1, 2, . . .

(means for changing luminance threshold)
A method for changing the luminance threshold Bth[n] will be described. In the present invention, it is detected whether or not the voltage V1 of the holding capacitor 132 exceeds the threshold. A voltage V1 of the holding capacitor 132 is expressed by the following equation.

Here, Vb0 is the charging voltage of the light receiving element 131, ΔV is the amount of voltage decrease due to the photocurrent flowing through the light receiving element 131, J is the current density of the photocurrent flowing through the light receiving element 131, S is the light receiving area of the light receiving element 131, and Δt is The exposure time of the light receiving element 131 and C is the capacitance of the holding capacitor 132 . In this embodiment, a photodiode is used as the light receiving element 131 .

There are the following methods for changing the luminance threshold Bth of the pixel 130 .
(A) Change the exposure time Δt.
(B) The area S of the light receiving element 131 is changed.
(C) The charge voltage Vb0 to the light receiving element 131 is changed.

In the present invention, any of the methods (A), (B), and (C) may be used to change the brightness threshold. Also, a plurality of these methods may be combined. Furthermore, the brightness threshold may be changed by methods other than (A) to (C).

That is, the imaging system 1 of this embodiment includes a brightness threshold value changing means. The luminance threshold changing means includes (A) changing the exposure time Δt for each sub-image, (B) providing a plurality of sub-pixels having different light-receiving areas of the light-receiving element 131, and (C) charging voltage Vb0 applied to the light-receiving element 131. is changed every predetermined period, and further includes means for combining these (A) to (C).

Moreover, the luminance threshold changing means described above is typically provided in the sensor section 10 . This is because it becomes easy to implement. However, it may be provided in a part other than the sensor part.

(Change the brightness threshold depending on the exposure time)
In this embodiment, the luminance threshold is changed by changing the exposure time.

FIG. 7 is a timing chart schematically showing a readout sequence from the pixels 130 in the sensor element 13 of this embodiment. In this figure, the horizontal axis indicates time, and the vertical axis indicates the scanning line position of the pixel 130 within the sensor element 13 . The fact that the diagram representing the scanning timing becomes a slanted straight line as shown in FIG. 7 indicates that the scanning lines are sequentially scanned in the order of the first scanning line, the second scanning line, and so on.

One field is the time to acquire one image, typically 1/30 second to 1/60 second. In this embodiment, it is set to 1/30 seconds (33.3 ms). One field is divided into four subfields. Get one sub-image in one sub-field. Although FIG. 7 shows an example in which four sub-images are acquired, it may of course be divided into more sub-fields. Let the nth subfield be denoted as SF[n].

Taking the first subfield SF[1] as an example, the readout sequence within the subfield will be described. When the charging scanning pulse is applied to the charging scanning line 135 of the first row, the light receiving element 131 of the pixel 130 of the first row is charged to the charging voltage Vb0. After that, charging scanning pulses are applied to the charging scanning lines 135 of the second and subsequent rows.

When a readout scanning pulse is applied to the readout scanning line 136 of the first row after the time Δtex has passed, the threshold determination circuit 134 of each pixel 130 operates, and based on the voltage of the storage capacitor 132 at that time, the output signal 1 Alternatively, 0 is output from the threshold decision circuit 134 . After that, a readout scanning pulse is applied to the readout scanning lines 136 of the second and subsequent rows. Thus, the time Δtex from the application of the charging scanning pulse to the application of the reading scanning pulse corresponds to the exposure time.

Thus, scanning of the first sub-field SF[1] is completed, and the first sub-image SIm[1] is acquired. Next, the charging scanning line 135 and the reading scanning line 136 are similarly scanned in the second subfield SF[2]. However, the elapsed time Δtex between the charging scanning line 135 and the reading scanning line 136 is made longer than SF[1]. Next, the elapsed time Δtex between the charging scanning line 135 and the reading scanning line 136 is made longer than SF[2], and the charging scanning line 135 and the reading scanning line 136 are similarly set in the third subfield SF[3]. scan. Further, the elapsed time Δtex between the charge scanning line 135 and the readout scanning line 136 is set longer than SF[3], and the charge scanning line 135 and the readout scanning line 136 are similarly set in the fourth subfield SF[4]. Scan. In this way, sub-images are obtained by changing the exposure time Δtex for each sub-field.

As can be seen from Equation (1), changing the exposure time Δtex changes the luminance threshold corresponding to the voltage threshold of the threshold determination circuit 134 . For the same object, the longer the exposure time Δtex, the lower the voltage of the storage capacitor 132. Therefore, even a lower luminance (dark luminance) exceeds the voltage threshold of the threshold determination circuit 134, and the output signal "1" is given. In this way, multiple sub-images with different brightness thresholds can be acquired.

(Structure of gradation image)
The plurality of acquired sub-images are transmitted from the sensor unit 10 to the image processing unit 20 as shown in FIG. In the image processing unit 20, a gradation image is constructed by the following method from a plurality of sub-images having different brightness thresholds.

FIG. 8 is a schematic diagram showing a method of forming a gradation image. FIG. 8 shows an example of forming a gradation image of 4 gradations from 4 sub-images.

FIG. 8A shows four sub-images. Each of the sub-images is a binary image representing the image of the subject. image, and the fourth image is a sub-image with a brightness threshold of 1. The hatched area in the figure is the pixel area of the output signal "1".

The higher the luminance threshold, the smaller the area of the output signal "1", ie the area above the luminance threshold. Also, the region of the output signal "1" in the sub-image with the lower luminance threshold includes the region of the output signal "1" in the sub-image with the higher luminance threshold.

Let S[n] be the image data of the n-th sub-image SIm[n]. Since the sub-image is a binary image, its image data S[n] is two-dimensional data in which each pixel 130 is assigned a signal "1" or "0". Consider a case where the brightness threshold Bth[n] increases as n decreases.

FIG. 9 is a logic table showing the process of constructing a grayscale image. A gradation image GS[n] is constructed from image data S[n] and S[n−1] of sub-images having adjacent luminance thresholds. The gradation luminance corresponding to the luminance threshold value Bth[n] is given to the pixel 130 having "1" in the column of GS[n]. As shown in FIG. 9, the gradation Bth[n] is given to the pixel 130 where S[n] is "1" and S[n−1] is "0". A gradation image GS[n] can be generated by such signal processing.

A logical expression of the signal processing in FIG. 9 is given by the following expression.

The signal processing of FIG. 9 can be implemented with the circuit of FIG. Whether or not the pixel 130 has the n-th gradation GS[n] is determined by a logical product (AND) of a signal obtained by logically inverting the sub-image data S[n−1] by an inverter and S[n]. You get results.

When this signal processing is performed for n=1 to 4, each pixel 130 is assigned a luminance gradation of 0 or one of GS[1] to GS[4], and a gradation image can be generated. In this way, signal processing between binary sub-images has the advantage that a gradation image can be generated with a simple processing circuit. Of course, the signal processing of the logic table of FIG. 9 may be mathematically processed using a computer or CPU (Central Processing Unit).

Another embodiment of generating a gradation image from a plurality of sub-images with different thresholds will be described with reference to FIGS. 11 and 12. FIG.

FIG. 11 is a schematic diagram showing a state in which the image data S[n] of the sub-image SIm[n] are stored in the memory. (k, m) in the figure corresponds to the pixel 130 at the position of k rows and m columns. As shown in FIG. 11, a plurality of sub-images S[n] are arranged on the memory so that pixels 130 correspond to each other, and the gradation of each pixel 130 is calculated by signal processing of each pixel 130 therefrom.

A truth table shown in FIG. 12 is used for the method of calculating the gradation image. A gradation level output GS is obtained according to the values of S[1] to S[4]. In the truth table of FIG. 12, "X" indicates that either "1" or "0" is acceptable. Also, Bth[n] in the output GS column means the brightness threshold of the sub-image S[n].

Processing based on the truth table of FIG. 12 can be easily implemented using an FPGA (Field Programmable Gate Array). Alternatively, it may be implemented by arithmetic processing by a CPU.

The gradation image generated as described above is displayed on the display device 21 connected to the image processing section 20 . Moreover, the gradation image is stored in the image recording section 22 connected to the image processing section 20 as necessary. The image recording section 22 may of course be incorporated into the image processing section 20 .

(Advantages of Separating Image Processing Unit 20)
In this embodiment, as shown in FIG. 1, the image processing section 20 and the sensor section 10 are separated and connected by a cable 30. FIG. Such a separate configuration has the effect of improving the radiation resistance.

The sensor unit 10 is installed in a high radiation dose environment, and the image processing unit 20 is installed in a low radiation dose environment. In FIG. 1, the area (area A) on the left side partitioned by the wall 40 is the high dose environment, and the area (area B) on the right side of the wall 40 is the low dose environment. In this way, the image processing unit 20 can be configured with a normal CPU and circuit parts having low radiation resistance, which is more preferable.

As a method of further improving the radiation resistance of the imaging system 1 of this embodiment, as shown in FIG. 1, the image processing section 20 may be surrounded by a radiation shielding material 25 such as a lead plate. Unlike the sensor section 10, the image processing section 20 does not lose its function even if it is surrounded by a light-impermeable member. By separating the image processing unit 20 and the sensor unit 10 in this way, there is an effect that the radiation resistance is further increased.

<Example 2>
An imaging system 1 according to a second embodiment of the present invention will be described with reference to FIGS. 13 and 14. FIG.

In the imaging system 1 of the present embodiment, the sensor element 13 in which the sub-pixels 1300 having different light receiving areas are provided in the pixels 130 is used in order to acquire sub-images with different brightness thresholds. In the second embodiment, the description of the configuration having the same function as that of the first embodiment is omitted, and mainly the different configuration will be described.

FIG. 13 is a diagram showing the configuration of the sensor element 13 of the imaging system 1 of this embodiment, showing four pixels 130 in the sensor element 13. As shown in FIG. In the drawing, a portion surrounded by a dashed line constitutes one pixel 130 . One pixel 130 is composed of four sub-pixels 1300, and the areas of the four sub-pixels 1300 are different from each other.

The configuration of each sub-pixel 1300 is the same as in FIG. However, the light-receiving area of the light-receiving element 131 of the sub-pixel 1300 differs depending on the sub-pixel 1300 . As shown in formula (1), when trying to obtain a predetermined amount of voltage decrease ΔV, a larger photocurrent density J is required to obtain a predetermined ΔV as the light-receiving area is smaller. Since the photocurrent density J corresponds to the brightness of the observed object, the smaller the light receiving area, the larger the brightness threshold Bth. The numbers written inside each pixel 130 in FIG. 13 are the relative values of the brightness threshold Bth of each sub-pixel 1300 . The luminance threshold changes according to the light receiving area of the light receiving element 131 .

Also, as can be seen from FIG. 13, one pixel 130 requires two charge scanning lines 135 and two readout scanning lines 136, so the number of lines to be scanned is double that of the first embodiment. In this embodiment, a configuration in which the storage capacitor 132 is provided separately from the light receiving element 131 is preferable. The reason is that the magnitude of the junction capacitance of the light receiving element 131 may be roughly proportional to the area of the light receiving element 131 . As can be seen from Equation (1), when the capacitance C is proportional to the area S, the luminance threshold does not change.

Although not shown, the pixels 130 may be provided with color filters. For example, a Bayer filter in which three colors of red (R), green (G), and blue (B) are arranged in an RGGB pattern can be used.

FIG. 14 is a timing chart schematically showing the readout sequence from the pixels 130 in the sensor element 13 of this embodiment. Since the luminance threshold is changed by the area of the sub-pixel 1300, four sub-images with different luminance thresholds can be obtained by scanning only once in one field period. For the same number of pixels, the number of scanning lines is doubled compared to the first embodiment.

The method of generating a gradation image from sub-images is the same as in the first embodiment.

(Sub-pixel arrangement pattern)
In addition, in the configuration of the pixel 130 in FIG. 13, the direction of extraction of the data line 137 is reversed between the sub-pixels 1300 so that the distance between the light-receiving elements 131 of the sub-pixels 1300 is shortened. As an example, the threshold 8 sub-pixel 1300 has the data line 137 extending to the left, and the threshold 2 sub-pixel 1300 has the data line 137 extending to the right. By adopting such an arrangement pattern, the light receiving elements 131 between the sub-pixels 1300 are brought closer to each other.

In this way, by adopting the arrangement of the sub-pixels 1300 in which the extraction direction of the data line 137 is opposite between the sub-pixels 1300, even when the luminance change of the observed object is spatially small, the gradation is not disturbed. It becomes difficult to generate, and an effect of reducing the so-called moire phenomenon is obtained.

<Example 3>
An imaging system 1 according to a third embodiment of the present invention will be described with reference to FIGS. 15 and 16. FIG.

In the imaging system 1 of the present embodiment, the charging voltage Vb0 of the light receiving element 131 is varied in order to acquire sub-images with different luminance thresholds. In the third embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

FIG. 15 is a schematic diagram showing the relationship between luminance, that is, the amount of light incident on the light receiving element 131 and the terminal voltage.

This figure shows the characteristics when the charging voltage Vb0 is changed to four values of Vb01 to Vb04.

As can be seen from the formula (1), when the brightness is zero, ΔV=0, so the terminal voltage V1 of the storage capacitor 132 is approximately equal to the charging voltage Vb0. Then, when light is incident and a photocurrent flows, the voltage decreases by ΔV according to the formula (1), so that the characteristics shown in FIG. 15 are obtained.

As can be seen from FIG. 15, when the charging voltage Vb0 is changed to four values of Vb01 to Vb04, the luminance changes when the terminal voltage V1 of the storage capacitor 132 becomes equal to the predetermined threshold value Vth. That is, the luminance threshold Bth changes. In this embodiment, this characteristic is used to obtain a plurality of sub-images with different brightness thresholds.

The configuration of the sensor element 13 used in this embodiment is the same as that shown in FIG.

FIG. 16 is a schematic diagram showing the relationship between the readout sequence from each pixel 130 and the charging voltage Vb0 in this embodiment.

One field is divided into four subfields SF[1] to SF[4]. In each subfield, the charging scanning pulse is sequentially scanned (timing indicated by the dotted line in the drawing), and the reading scanning pulse is sequentially scanned after the exposure time Δtex has elapsed. A feature of this embodiment is that the charging voltage Vb0 is changed for each subfield. Thereby, sub-images with different brightness thresholds can be acquired.

The method of generating a gradation image from sub-images is the same as in the first embodiment.

<Example 4>
An imaging system 1 according to a fourth embodiment of the present invention will be described. The present embodiment is a hybrid type that uses a plurality of methods for changing the brightness threshold value described in Equation (1). As a result, there is an effect that an image with a higher number of gradations can be obtained. In the fourth embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

In this embodiment, the configuration of the sensor element 13 shown in FIG. 13 is used, and the charging voltage is changed for each subfield in the scanning sequence shown in FIG. As an example, using the sensor element 13 composed of four sub-pixels 1300 as shown in FIG. 13, a four-gradation image can be obtained. Then, as shown in FIG. 16, when sub-images are obtained by changing the charging voltage Vb0 in four stages, an image with four gradations can be obtained. As a result, a 16-gradation image of 4×4 can be generated.

<Example 5>
An imaging system 1 according to a fifth embodiment of the present invention will be described with reference to FIG. The present embodiment is a hybrid type that uses a plurality of methods for changing the brightness threshold value described in Equation (1). In this embodiment, the change in exposure time and the change in charging voltage Vb0 are combined. In the fifth embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

As shown in FIG. 17, the first field includes four subfields SF[1]-SF[4] with different exposure times. In the second field, after changing the charging voltage Vb0, an image is acquired in four subfields with different exposure times. Thus, eight sub-images with different brightness thresholds are acquired. From this sub-image, an 8-gradation image can be generated by the method described above.

Although description is omitted, sub-pixels 1300 having different pixel areas and changes in exposure time may be combined as a third aspect of the hybrid type.

<Example 6>
An imaging system 1 according to a sixth embodiment of the present invention will be described. The imaging system 1 of this embodiment can operate in two imaging modes, a normal mode and a high gradation mode. In the sixth embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

The imaging system 1 of the present embodiment obtains a 4-gradation image by the driving timing shown in FIG. 7 when real-time performance is required. This is called normal mode. On the other hand, when an image with higher gradation is required, the mode is changed to the high gradation mode. In the high gradation mode, the drive timing shown in FIG. 17 is used. That is, one field is divided into subfields with different exposure times, and the charging voltage Vb0 is changed for each field, and an image is captured over n field periods. Although the case of n=2 is shown in FIG. 17, n=8 may be used. When n=8, the time required for imaging is eight times longer, but a gradation image with 4×8=32 gradations can be generated.

In addition, although the case where the gradation image in the normal mode has 4 gradations is shown, this is only an example, and the normal mode may have 16 gradations, for example.

The point of this embodiment is that the number of sub-images used to generate a gradation image is greater in the high gradation mode than in the normal mode. Accordingly, in the high gradation mode, a high gradation image can be captured.

<Example 7>
A robot system will be described with reference to FIG. 18 as an example of imaging system application equipment according to the seventh embodiment of the present invention.

Applied equipment of this embodiment is a robot system that can be used even in a severe environment such as a high-level radiation environment. In the seventh embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

The robot system of this embodiment has a robot 100 and a robot controller 200 . The robot 100 has a robot stand 101 on which tires 102 are attached, an arm 110 and a sensor section 10 . The arm 110 has joints 111 and can perform actions such as rotation and bending. Although not shown, a mechanism that allows the arm 110 to extend and contract may be provided. The sensor section 10 has the sensor structure of any of the embodiments described above.

The robot control device 200 is a control device that can operate the robot 100 by remote control. The robot control device 200 has an image processing section 20 and a robot control section 210 . The sensor unit 10 and the image processing unit 20 are connected by a cable 30 and transmit and receive signals to and from each other. The robot 100 and the robot controller 210 are connected by a cable 31 to supply power and transmit/receive control signals for driving the robot 100 . The image processing unit 20 and the robot control unit 210 transmit and receive signals as necessary.

In this embodiment, since the sensor unit 10 has excellent radiation resistance, the robot 100 can capture images even in a high-level radiation environment. In particular, in the present embodiment, the robot 100 (sensor unit 10) is arranged in the area A (left side of the wall 40 in FIG. 18) which is a high level radiation environment, and the robot 100 (sensor unit 10) is arranged in the area B (right side of the wall 40) which is a low radiation environment. An image processing unit 20 is arranged. This allows the image processing unit 20 to operate in a low radiation environment.

Moreover, the sensor element 13 constituting the sensor section 10 of the present embodiment is small and lightweight compared to the case of using a vacuum tube type image pickup element such as an image pickup tube. Therefore, the robot 100 can be installed in a narrow space, and even if a plurality of sensor units 10 are installed in the robot 100 , an increase in weight can be suppressed, and the impact on the installation space can be reduced. As a result, the object 300 can be grasped more accurately, and the robot 100 can be made to perform sophisticated motions.

Also, in the robot system of this embodiment, it is preferable to employ a configuration capable of switching between the high gradation mode and the normal mode, as described in the sixth embodiment, as required. When the robot 100 is moving or when the target object 300 is operated by the arm 110, the sensor section 10 operates in the normal mode. As a result, the image of the target object 300 can be acquired at high speed, so that high-speed feedback can be provided to the robot control unit 210 . On the other hand, when photographing the object 300 and surrounding images for recording, the sensor unit 10 is operated in the high gradation mode. As a result, high-quality images can be recorded with a large number of gradations.

<Example 8>
An imaging system 1 according to an eighth embodiment of the present invention will be described with reference to FIGS. 19 and 20. FIG.

In this embodiment, the threshold determination circuit 134 is not provided in the pixel 130 but is provided for each data line 137 . In the eighth embodiment, explanations of configurations having the same functions as those of the above-described embodiments will be omitted, and mainly different configurations will be explained.

FIG. 19 is a schematic diagram showing the configuration of the pixel 130 of the sensor element 13 of this embodiment.

A terminal of the holding capacitor 132 is connected to the data line 137 through the read transistor 143 .

FIG. 20 is a schematic diagram showing the configuration of the sensor element 13 of the imaging system 1 of the present embodiment, and shows four pixels 130 in the sensor element 13, but actually more pixels are arranged. .

Each data line 137 is connected to the input terminal IN of the threshold determination circuit 134 . An output terminal O of the threshold determination circuit 134 is connected to the transmission/reception section 12 via a switch 141 .

A signal reading procedure in this embodiment will be described. In this embodiment, scan pulses are applied at the timings shown in FIG. When the charging scanning pulse is applied to the charging scanning line 135, the charging scanning pulse is applied to the CA terminals of the pixels (FIG. 19) on the selected scanning line, and the charging transistor 133 is turned on. Thereby, the light receiving element 131 is charged to the charging voltage Vb0. After that, the charging transistor 133 is turned off.

After a predetermined exposure time, when a readout scanning pulse is applied to the readout scanning line 136, the readout transistor 143 of the selected pixel is turned on. At this time, the readout transistors 143 of the pixels on the unselected scanning lines are in the OFF state. Therefore, the voltage of the storage capacitor 132 of the pixel on the selected scanning line is output to the data line 137. FIG.

At this point, a read pulse is output from the read pulse generator 144 shown in FIG. As a result, the output terminal O of the threshold determination circuit 134 is latched to the signal "1" or "0" based on the magnitude relationship between the voltage of the data line 137 at the time of application of the read pulse and the predetermined threshold.

During this period, horizontal scanning pulses sequentially output from the horizontal scanning circuit 140 operate the switch 141 , and the signal voltage (“1” or “0”) of each data line 137 is transferred to the transmitting/receiving section 12 . Since a binary signal flows through each data line 137 after the output of the threshold determination circuit 134, the switch 141 may be a logic circuit.

Thus, binary sub-image data is obtained based on the voltage of the storage capacitor 132 of each pixel. As shown in FIG. 7, sub-images with different brightness thresholds can be obtained by changing the exposure time for each sub-field. The method of generating a gradation image from a plurality of sub-images is as described above in the first embodiment.

In FIG. 19, a MOSFET is used as the read transistor 143. In this operation, the operation is performed in the saturation region of the FET. less impact. Therefore, radiation resistance is improved as compared with the case of using a CMOS type amplifier.

According to this embodiment, the circuitry within the pixel 130 can be simplified.

In this embodiment, the brightness threshold value is changed by changing the exposure time, but a structure in which the charging voltage is changed (FIG. 16), a structure in which the area of the light receiving element 131 is changed (FIG. 13), or a hybrid type in which a plurality of methods are combined. ( FIG. 17 , etc.) can also apply the configuration in which the threshold determination circuit 134 is provided for each data line 137 .

As described above, the imaging system 1 of the embodiment of the present invention includes the sensor section 10 and the image processing section 20. The sensor section 10 includes the transmission/reception section 12 and the sensor element 13, and the sensor element 13 has a light receiving element 131 and a charging transistor 133 for each pixel 130, the sensor element 13 has one or a plurality of threshold determination circuits 134 for determining whether the terminal voltage of the light receiving element 131 is equal to or higher than a predetermined threshold, The sensor element 13 acquires a plurality of sub-images corresponding to different luminance thresholds, and the image processing unit 20 processes the signals of the plurality of sub-images to generate a gradation image. Therefore, it is possible to provide a compact and lightweight imaging device that can be used even in a harsh environment such as a high-level radiation environment.

In the first embodiment, since the sensor element 13 has the threshold determination circuit 134 for each pixel 130, the sensor element 13 can operate reliably even if the number of scanning lines is large. You can control the impact.

Further, since the sensor element 13 changes the luminance threshold by changing the exposure time of the light receiving element 131 for each sub-image, there is no need to provide a sub-pixel within the pixel, and the pixel configuration can be simplified.

In addition, in the second embodiment, each pixel 130 of the sensor element 13 includes a plurality of sub-pixels 1300 having different light receiving areas of the light receiving element 131. Therefore, one image can be acquired by one scanning, and one pixel can be read out. can extend the time.

In addition, in the third embodiment, the sensor element 13 changes the charging voltage applied to the light receiving element 131 every predetermined period. Therefore, it is not necessary to provide a sub-pixel within the pixel, and the pixel configuration can be simplified. . Also, by changing the voltage each time the sub-pixel is acquired, the luminance threshold of the sub-pixel can be changed. In addition, by changing the voltage for each cycle in which a plurality of sub-pixels are acquired, it is possible to collectively change the luminance thresholds of the plurality of sub-pixels.

In another embodiment, the sensor unit 10 and the image processing unit 20 are spatially separated and connected by wire or wirelessly, so that the imaging apparatus can be used even in harsh environments such as high-level radiation environments. can provide Also, an inexpensive and high-performance semiconductor device can be used for the image processing unit 20 .

In another embodiment, since the image processing unit 20 is surrounded by the radiation shielding material 25, it is possible to provide an imaging apparatus that can be used even in harsh environments such as high-level radiation environments. Also, an inexpensive and high-performance semiconductor device can be used for the image processing unit 20 .

Further, in another embodiment, the threshold determination circuit 134 has the bipolar transistor Tr2 to which the output voltage of the light receiving element 131 is input, so that the radiation resistance performance can be improved.

Also, in another embodiment, the imaging system 1 is operable in a normal mode and a high grayscale mode, and the number of sub-images acquired to generate a grayscale image in the high grayscale mode is scaled in the normal mode. Since the number of sub-images is larger than the number of sub-images acquired for generating a tone image, it is possible to obtain an image giving priority to either real-time performance or high image quality according to the purpose of image acquisition.

It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the described configurations. Also, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Moreover, the configuration of another embodiment may be added to the configuration of one embodiment. Further, additions, deletions, and replacements of other configurations may be made for a part of the configuration of each embodiment.

In addition, each configuration, function, processing unit, processing means, etc. described above may be realized by hardware, for example, by designing a part or all of them with an integrated circuit, and the processor realizes each function. It may be realized by software by interpreting and executing a program to execute.

Information such as programs, tables, and files that implement each function can be stored in storage devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC cards, SD cards, DVDs, and BDs. can.

In addition, the control lines and information lines indicate those considered necessary for explanation, and do not necessarily indicate all the control lines and information lines necessary for mounting. In practice, it can be considered that almost all configurations are interconnected.

1 imaging system 10 sensor unit 11 sensor control unit 12 transmission/reception unit 13 sensor element 20 image processing unit 21 display device 22 image recording unit 25 shielding material 30, 31 cable 100 robot 101 robot table 102 tire 110 arm 111 joint 130 pixel 131 light receiving element 132 storage capacitor 133 charge transistor 134 threshold determination circuit 135 charge scanning line 136 readout scan line 137 data line 138 charge scan circuit 139 readout scan circuit 140 horizontal scan circuit 143 readout transistor 144 readout pulse generator 200 robot controller 210 robot controller 300 Object 1300 sub-pixel

Claims (11)

An imaging system comprising a sensor unit and an image processing unit,
The sensor unit has a transmitting/receiving unit and a sensor element,
The sensor element has a light receiving element and a charging transistor for each pixel,
The sensor element has one or more threshold determination circuits for determining whether the terminal voltage of the light receiving element is equal to or higher than a predetermined threshold,
The sensor element changes the luminance threshold by changing the exposure time of the light receiving element for each sub-image,
the sensor element acquires a plurality of sub-images corresponding to different luminance thresholds;
The imaging system, wherein the image processing unit processes signals of the plurality of sub-images to generate a gradation image. An imaging system comprising a sensor unit and an image processing unit,
The sensor unit has a transmitting/receiving unit and a sensor element,
The sensor element has a light receiving element and a charging transistor for each pixel,
The sensor element has one or more threshold determination circuits for determining whether the terminal voltage of the light receiving element is equal to or higher than a predetermined threshold,
the sensor element acquires a plurality of sub-images corresponding to different luminance thresholds;
each pixel of the sensor element includes a plurality of sub-pixels having different light receiving areas of the light receiving element,
The imaging system, wherein the image processing unit processes signals of the plurality of sub-images to generate a gradation image. An imaging system comprising a sensor unit and an image processing unit,
The sensor unit has a transmitting/receiving unit and a sensor element,
The sensor element has a light receiving element and a charging transistor for each pixel,
The sensor element has one or more threshold determination circuits for determining whether the terminal voltage of the light receiving element is equal to or higher than a predetermined threshold,
The sensor element changes a charging voltage applied to the light receiving element every predetermined period,
the sensor element acquires a plurality of sub-images corresponding to different luminance thresholds;
The imaging system, wherein the image processing unit processes signals of the plurality of sub-images to generate a gradation image. An imaging system according to any one of claims 1 to 3,
The imaging system, wherein the sensor element has the threshold determination circuit for each pixel. An imaging system according to any one of claims 1 to 3,
An imaging system, wherein the sensor unit and the image processing unit are spatially separated and connected by wire or wirelessly. An imaging system according to claim 5, wherein
The imaging system, wherein the image processing unit is surrounded by a radiation shielding material. An imaging system according to any one of claims 1 to 3,
The imaging system, wherein the threshold determination circuit has a bipolar transistor to which the output voltage of the light receiving element is input. An imaging system according to any one of claims 1 to 3,
operable in normal mode and high gradation mode,
The imaging system, wherein the number of sub-images acquired to generate the gradation image in the high gradation mode is greater than the number of sub-images acquired to generate the gradation image in the normal mode. . An imaging system application device comprising the imaging system according to any one of claims 1 to 8. The imaging system application equipment according to claim 9,
a radiation-resistant device section having the sensor section;
An image pickup system application device, comprising: a control device having the image processing section and a device control section for controlling the operation of the image pickup system application device. The imaging system application equipment according to claim 9,
a robot on which the sensor unit is installed;
An imaging system application apparatus, comprising: a control device having the image processing section and a robot control section for controlling the operation of the robot.

Images