JP2018195991A - Imaging element, method for controlling imaging element, imaging device, and electronic apparatus - Google Patents

Abstract

To realize reduction in the influence of vertical line noise caused by noise generated in fixedly arranged column AD circuits.SOLUTION: Every time a pixel signal is output in units of rows, connections between vertical transmission lines and an AD converter are switched in units of columns for connection and, in the connection switched state, the pixel signal that is AD-converted by the AD converter is read after being restored to the pixel signal before the connections were switched. The present disclosure may be applied to imaging devices.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an imaging element, an imaging element control method, an imaging apparatus, and an electronic device, and more particularly, an imaging element that can reduce the influence of vertical stripe noise, an imaging element control method, an imaging apparatus, and an electronic device Regarding equipment.

  In a conventional column AD (Analog to Digital Convert) type CMOS (Complementary Metal Oxide Semiconductor) image sensor, pixels arranged in the same column are read out by the column AD circuit in the same column, so a fixed column AD circuit in a predetermined column is generated. When the pattern noise or random noise is larger than that of the column AD circuit of another column, noise having a correlation in the column direction is generated in the pixel signal of the predetermined column, and is easily visually recognized as vertical stripe noise.

  In view of this, a technique has been proposed in which a switch is provided between the pixel and the sample hold circuit, and the visual connection of the noise in the column direction is reduced by switching the connection of the switches at random.

Special table 2008-537406 gazette

  However, in order to effectively reduce visual recognition, connection switching in units of several tens of columns is necessary, and the configuration of the switch becomes complicated.

  Furthermore, in order not to make the connection switching pattern constant, it is necessary to add a circuit for generating a random number for making the selection of the pattern random, and the entire system becomes large-scale and the cost as well as the chip size is increased. Increase.

  The present disclosure has been made in view of such a situation, and in particular, reduces the influence of vertical streak noise caused by noise generated in a fixed column AD circuit.

  An imaging device according to one aspect of the present disclosure includes a pixel array that outputs pixel signals corresponding to the amount of incident light in units of pixels arranged in an array, and a plurality of vertical units that transfer the pixel signals in units of rows for each column. A connection between the transfer line, a column AD conversion unit having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signals for each column, the vertical transfer line, and the AD converter is connected to the vertical converter. The image sensor includes a connection switching unit that shifts and switches a predetermined number of columns each time the pixel signal is transferred in units of rows by a transfer line.

  The pixel in the original column before the connection is switched with the pixel signal AD-converted by the AD converter in a state where the connection is shifted by the connection switching unit and the connection is switched. A reading unit that reads out as a signal can be further included.

  The connection switching unit may further include a storage unit that stores the pixel signal AD-converted by the AD converter in a state where the connection is switched by shifting the predetermined number of columns. In the readout unit, pixels that are AD-converted by the AD converter in a state in which the connection is switched by shifting the predetermined number of columns by the connection switching unit stored in the storage unit The signal can be read out as a pixel signal of the original column before the connection is switched.

  The predetermined number of columns may be changed every time the pixel signal is transferred in units of rows by the vertical transfer line.

  The predetermined number of columns may be fixedly changed with a predetermined number of times that the pixel signal is transferred in units of rows by the vertical transfer line.

  The connection switching unit connects the vertical transfer line and the AD converter in the first direction by the predetermined number of columns when the pixel signal is transferred in units of rows by the vertical transfer line. After shifting and switching the connection, when the pixel signal is transferred next time, the predetermined number of columns shifts in the second direction to switch the connection, and then the pixel signal is transferred. The connection can be repeated without shifting.

The predetermined number of columns may be 2 ^k (k is a natural number).

  The readout unit may further include a filter unit that performs a filtering process on the pixel signal that is restored and read out as the pixel signal before the connection is switched.

  The filter unit may be a moving average filter or a median filter.

  The connection switching unit includes an output terminal of the vertical transfer line in units of columns, an input terminal of the AD converter in units of columns, the output terminal, and a connection selector switch that switches connections between the input terminals, Each time a pixel signal is output in units of rows, the connection selector switch is controlled to switch the connection between the output terminal and the input terminal. it can.

  A first conduction switch for switching on or off the connection between the output terminal and the input terminal of the column shifted in the first direction by the predetermined number of columns; the output terminal; and the predetermined number of columns The second conduction switch for switching on or off the connection with the input terminal of the column shifted in the second direction, and the connection between the output terminal and the input terminal of the same column on or off. A third conduction switch to be switched can be included, and the connection switching unit controls whether the first to third conduction switches are turned on or off, so that the pixel signal is output in units of rows. Each time it is output, the connection switch can be controlled to switch the connection between the output terminal and the input terminal.

  The column AD circuit can include the same number of dummy AD converters as the number of columns to be shifted at the column end, and the connection switching unit includes the column unit at the column end, The same number of dummy connections as the number of columns to be shifted are switched to connect the dummy input terminals of the dummy AD converters as many as the number of columns to be shifted, the vertical transfer lines at the column ends, and the dummy input terminals. When the connection between the vertical transfer line and the AD converter is shifted and switched by the predetermined number of columns, the dummy connection switch is controlled, The vertical transfer line at the column end and the dummy input terminal at the column end are connected, and the dummy AD converter performs AD conversion on a pixel signal transferred through the vertical transfer line at the column end. Can be.

  The column end can further include a dummy output terminal of a dummy vertical transfer line having a power supply potential or a ground potential, and the connection between the vertical transfer line and the AD converter is the same. The connection switching unit can control the dummy connection changeover switch so that the dummy output terminal and the dummy input terminal are connected to each other.

  An image sensor control method according to an aspect of the present disclosure includes a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array, and transfers the pixel signal in rows for each column. An image sensor control method, comprising: a plurality of vertical transfer lines; and a column AD conversion unit having a plurality of AD converters that perform AD (Analog to Digital) conversion of the pixel signals for each column. And a step of shifting the connection with the AD converter by a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line.

  An imaging device according to one aspect of the present disclosure includes a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array, and a plurality of vertical units that transfer the pixel signals in units of rows for each column. A connection between the transfer line, a column AD conversion unit having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signals for each column, the vertical transfer line, and the AD converter is connected to the vertical converter. The image pickup apparatus includes a connection switching unit that shifts and switches a predetermined number of columns each time the pixel signal is transferred in units of rows by a transfer line.

  An electronic apparatus according to one aspect of the present disclosure includes a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array, and a plurality of vertical units that transfer the pixel signals in units of rows for each column. A connection between the transfer line, a column AD conversion unit having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signals for each column, the vertical transfer line, and the AD converter is connected to the vertical converter. An electronic apparatus including a connection switching unit that shifts and switches a predetermined number of columns each time the pixel signal is transferred in units of rows by a transfer line.

  In one aspect of the present disclosure, a plurality of vertical transfer lines that transfer pixel signals in units of rows for each column and a plurality of AD converters that perform AD (Analog to Digital) conversion of the pixel signals for each column However, each time the pixel signal is transferred in units of rows by the vertical transfer line, the pixel signal is switched by a predetermined number of columns.

  According to one aspect of the present disclosure, it is possible to reduce the influence of vertical stripe noise caused by noise generated in a fixed column AD circuit.

It is a figure explaining the structural example of the image pick-up element of this indication. It is a figure explaining the concept of the connection switching part of FIG. It is a figure explaining the structural example of an image pick-up element in case the connection switching part of FIG. 1 is not provided. It is a figure explaining the principle which the vertical stripe noise generate | occur | produces by the image sensor of FIG. It is a figure explaining the principle which the vertical stripe noise generate | occur | produces by the image sensor of FIG. It is a figure explaining the connection switching pattern by a connection switching part. It is a figure explaining the connection switching pattern by a connection switching part. It is a figure explaining the connection switching pattern by a connection switching part. It is a figure explaining the example of the pixel signal read by switching a connection switching pattern by a connection switching part. It is a figure explaining the example which decompress | restores the pixel signal read as FIG. 9 shows. It is a figure explaining the detailed structural example of a connection switching part. It is a figure explaining the structure of the edge part of a connection switching part. It is a figure explaining the structure of the edge part of a connection switching part. It is a figure explaining the structure of the edge part of a connection switching part. It is a figure explaining the example of the pixel signal read by the image sensor in which the connection switching part is not provided. It is a figure explaining the example which applied the moving average filter to the pixel signal of FIG. It is a figure explaining the example which applied the median filter to the pixel signal of FIG. It is a figure explaining the example of the pixel signal read by the image sensor of FIG. It is a figure explaining the example which applied the moving average filter to the pixel signal of FIG. It is a figure explaining the example which applied the median filter to the pixel signal of FIG. 3 is a flowchart for explaining image processing by the image sensor of FIG. 1. It is a block diagram which shows the structural example of the imaging device as an electronic device to which the camera module of this indication is applied. It is a figure explaining the usage example of the camera module to which the technique of this indication is applied. It is a block diagram which shows an example of a schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of a vehicle exterior information detection part and an imaging part.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. 1. Image sensor of the present disclosure 2. Configuration example of connection switching unit 3. Detailed configuration example of connection switching unit 4. Configuration example of end of connection switching unit Filter processing Imaging processing Application example to electronic equipment8. 8. Usage example of image sensor Application examples

<1. Image Sensor of Present Disclosure>
FIG. 1 is a block diagram illustrating a configuration example of an imaging element according to the present disclosure. The image sensor of FIG. 1 reduces the influence of vertical stripe noise caused by noise generated in a column AD circuit in a specific column.

  1 includes a pixel array 31, a connection switching unit 32, a column AD circuit 33, vertical transfer lines 34-1 to 34-n, a memory 35, a reading unit 36, and a filter 37.

  The pixel array 31 generates a pixel signal corresponding to the amount of incident light in units of pixels 51 (FIG. 2) arranged in an array, and the vertical transfer line 34-for each column in units of rows. 1 to 34-n to the connection switching unit 32. Therefore, in the example of FIG. 1, the pixel array 31 has pixels 51 arranged in an array of n columns in the horizontal direction.

  Note that the vertical transfer lines 34-1 to 34-n are simply referred to as a vertical transfer line 34 unless otherwise distinguished, and the other configurations are also referred to in the same manner.

  The connection switching unit 32 connects the vertical transfer lines 34-1 to 34-n to the AD converter 33a (FIG. 2) provided for each column of the column AD circuit 33 for every column. Switch to shift left and right by a predetermined number of columns. The connection switching pattern between the vertical transfer line 34 and the column AD converter 33a (FIG. 2) by the connection switching unit 32 is different for each row, but is fixed at a predetermined number of rows and is repeated sequentially. The configuration of the connection switching unit 32 will be described later in detail with reference to FIG.

  The column AD (Analog to Digital Convert) circuit 33 includes an AD converter 33a (FIG. 2) for each vertical transfer line 34, that is, for each column. The AD converter 33a in each column converts the pixel signal of the analog signal supplied in units of rows through the vertical transfer line 34 connected from the connection switching unit 32 into a digital signal, and corresponds to the information in each column. In addition, the data is stored in a memory 35 composed of SRAM (Static Random Access Memory).

  The reading unit 36 is associated with the column corresponding to the vertical transfer line 34 in which the connection switching unit 32 shifts the predetermined number of columns for each row and the connection with the AD converter 33a (FIG. 2) is switched. Then, the pixel signal stored in the memory 35 is read out while restoring the pixel signal in the original column, and is output to the filter 37.

  That is, the pixel signal stored in the memory 35 is stored in a state where it is switched by the connection switching unit 32 in units of rows and shifted by a predetermined number of columns and converted into a digital signal by the column AD circuit 33. ing. When reading out the pixel signal from the memory 35, the reading unit 36 restores the pixel signal of the original column that has not been shifted and outputs it to the filter 37.

  The filter 37 performs filter processing such as a moving average filter and a median filter on the pixel data restored to the original pixel column, and outputs the image data.

<2. Configuration example of connection switching unit>
Here, a configuration example of the connection switching unit 32 will be described with reference to FIG. In FIG. 2, an example using 16 vertical transfer lines from the vertical transfer lines 34-1 to 34-16 will be described, but it goes without saying that the number of columns may be other than that.

  As shown in FIG. 2, the connection switching unit 32 includes terminals 32 a-1 to 32 a-16 on the pixel array 31 side for each of the vertical transfer lines 34-1 to 34-16, and the column AD circuit 33. Terminals 32b-1 to 32b-16 are provided on the side. Further, the connection switching unit 32 includes switches 32c-1 to 32c-16 that switch the connection between the terminals 32a-1 to 32a-16 and 32b-1 to 32b-16.

  In FIG. 2, the switch 32c-1 connects the terminal 32a-1 and the terminal 32b-1, the switch 32c-2 connects the terminal 32a-2 and the terminal 32b-2, and the switch 32c. −16 connects the terminal 32a-16 and the terminal 32b-16.

  In addition, in order not to make the description of a code | symbol too complicated, the code | symbol of the terminal 32a-3 thru | or 32a-15, the terminal 32b-3 thru | or 32b-15, and switch 32c-3 thru | or 32c-15 is abbreviate | omitted. Yes. In the following, in the case where notation of all symbols may be complicated, the description of the symbols is omitted as appropriate.

  In FIG. 2, the column AD circuit 33 includes AD converters 33a-1 to 33a-16 corresponding to the vertical transfer lines 34-1 to 34-16, respectively.

  For example, when the connection switching unit 32 continues to maintain the switches 32c-1 to 32c-16 in the connection state of FIG. 2, the configuration is substantially the same as the configuration without the connection switching unit 32 as shown in FIG. become.

  Here, as shown in FIG. 2 and FIG. 3, it is assumed that the AD converter 33 a-8 generates noise at a level larger than that of the other AD converters 33 a due to some factor. 2 and 3, the AD converter 33 a-8 is indicated by a hatched portion, which indicates that noise is generated.

  At this time, the pixel signals a, b, c, d, e, f, g, h, i, j, k, l, m are obtained in units of rows for each column from the vertical transfer lines 34-1 to 34-16. , n, o, p are supplied to the pixel converters a, b, c, d, e, f, g, h, i, j, in the AD converters 33-1 to 33-16 of the column AD circuit 33. If k, l, m, n, o, and p are AD-converted, the pixel signals in the n-th row, n + 1-th row, n + 2-th row,... are shown in FIG. Is converted to

  That is, in FIG. 4, the uppermost row is the pixel signal a, b, c, d, e, f, g, h, i output from each of the AD converters 33a-1 to 33a-16 in the nth row. , j, k, l, m, n, o, p. The second row from the top is the pixel signals a, b, c, d, e, f, g, h, i output from the AD converters 33a-c1 to 33a-16 in the (n + 1) th row. , j, k, l, m, n, o, p. Further, the third row from the top is the pixel signals a, b, c, d, e, f, g, h, i output by the AD converters 33a-1 to 33a-16 in the n + 2 row, respectively. , j, k, l, m, n, o, p. For the subsequent rows, the pixel signal in the n + 3 row, the pixel signal in the n + 4 row,..., The pixel signal in the n + 7 row are sequentially displayed.

  Here, since the pixel signal h in the eighth column is always output from the AD converter 33a-8, noise having a level higher than that of the other AD converters 33a is repeatedly added. In FIG. 4, pixel signals including noise are hatched.

  As a result, for all the rows, only the pixel signal in the eighth column is greatly affected by noise. For example, as shown in the range surrounded by the dotted line in FIG. The same color development as the pixels in the other columns is not performed, and it becomes easy to recognize as so-called vertical stripe noise.

  Therefore, the connection switching unit 32 controls the switches 32c-1 to 32c-16 to set the connection between the terminals 32a-1 to 32a-16 and the terminals 32b-1 to 32b-16 for each row. Switch to shift left and right by the number of columns.

  For example, when the pixel signal of the n-th row is output, as shown in FIG. 6, the connection switching unit 32 controls the switches 32c-1 to 32c-16 to shift by 0 columns, The terminals 32a-m and 32b-m are connected, and the pixel signal output from the vertical transfer line 34 is output to the AD converter 33a in the original column.

  Further, when the pixel signal of the (n + 1) th row is output, as shown in FIG. 7, the connection switching unit 32 controls the switches 32c-1 to 32c-16 to be connected to the terminals 32a-m. By connecting the terminal 32b- (m-4), the pixel signal output from the vertical transfer line 34 is shifted and output to the left AD converter 33a by four columns.

  Further, when the pixel signal of the (n + 2) th row is output, as shown in FIG. 8, the connection switching unit 32 controls the switches 32c-1 to 32c-16 to be connected to the terminals 32a-m. By connecting the terminal 32b- (m + 4), the pixel signal output from the vertical transfer line 34 is shifted to the right AD converter 33a by four columns and output.

  After that, the connection switching pattern for the n-th row, the connection switching pattern for the n + 1-th row, and the connection switching pattern for the n + 2-th row are fixedly repeated at a cycle of 3 rows. Here, m is 1 to 16, and is an identifier for identifying a column.

  More specifically, for example, in the n-th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16 so that the switch 32c-1 is connected to the terminal 32a-1 as shown in FIG. And the terminal 32b-1, the switch 32c-2 connects the terminal 32a-2 and the terminal 32b-2,..., The switch 32c-16 connects the terminal 32a-16 and the terminal 32b-16. Switch to connect. 6 is similar to the connection in FIG. 2, and is the same connection as in the case where the connection switching unit 32 is not provided.

  In the (n + 1) th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16 so that the switch 32c-5 is connected to the terminals 32a-5 and 32b as shown in FIG. −1, the switch 32c-6 connects the terminal 32a-6 and the terminal 32b-2,..., And the switch 32c-16 connects the terminal 32a-16 and the terminal 32b-12. To. That is, the connection is switched so that the pixel signal of the original vertical transfer line 34 is AD-converted by the AD converter 33a shifted to the left by four columns with respect to all columns. In FIG. 7, the connections of the terminals 32a-1 to 32a-4 and the terminals 32b-13 to 32b-16 are omitted.

  Further, in the (n + 2) th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16, so that the switch 32c-1 is connected to the terminal 32a-1 and the terminal 32b as shown in FIG. -5, the switch 32c-2 connects the terminal 32a-2 and the terminal 32b-6, and so on ... the switch 32c-12 connects the terminal 32a-12 and the terminal 32b-16. To. That is, the connection is switched so that the pixel signal of the original vertical transfer line 34 is AD-converted by the AD converter 33a shifted to the right by 4 columns with respect to all columns. In the right part of FIG. 2, the connections of the terminals 32a-13 to 32a-16 and the terminals 32b-1 to 32b-4 are omitted.

  That is, for the n-th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16 to convert all the pixel signals in units of the vertical transfer lines 34 with the AD converter 33a of the column as it is. Connect as you want. For the (n + 1) th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16 so that all the pixel signals are placed on the left side of the four columns in the figure with respect to the vertical transfer line 34. The shifted AD converter 33a is connected to perform AD conversion. Further, for the (n + 2) th row, the connection switching unit 32 controls the switches 32c-1 to 32c-16 so that all the pixel signals are placed on the right side of the four columns in the figure with respect to the vertical transfer line 34. The shifted AD converter 33a is connected to perform AD conversion.

  As a result, the pixel signals a, b, c, d, e, f, g, h, i, j, k, l, m, n, o for each column from the vertical transfer lines 34-1 to 34-16, respectively. , p are supplied, the n-th row is AD-converted by the AD converters 33a-1 to 33a-16 of the column AD circuit 33 as shown in the uppermost row of FIG. Pixel signals output from the 16th column are pixel signals a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, and p.

  The n + 1th row is AD-converted by the AD converters 33a-1 to 33a-12 of the column AD circuit 33 as shown in the second row from the top of FIG. Pixel signals output from the columns are pixel signals e, f, g, h, i, j, k, l, m, n, o, and p. Since the AD converters 33a-13 to 33a-16 are not connected, the output pixel signals are also omitted.

  Further, the n + 2 row is AD-converted by the AD converters 33a-5 to 33-16 of the column AD circuit 33 as shown in the third row from the top in FIG. Pixel signals output from the columns are pixel signals a, b, c, d, e, f, g, h, i, j, k, and l. Since the AD converters 33a-1 to 33a-4 are not connected, the output pixel signals are also omitted.

  For the n + 3th and subsequent rows in FIG. 9, pixel signals having the same connection switching pattern as those in the nth to n + 2th rows are fixedly and repeatedly output in a cycle of 3 rows. That is, the nth to n + 2 rows are the first connection switching pattern of the three-row cycle, and the n + 3 to n + 5 rows are the second connection switching pattern of the same three-row cycle. And so on.

  That is, as described above, since the AD converter 33a-8 of the column AD circuit 33 outputs a signal including noise, the pixel signal in the eighth column is the pixel signal h, l, d, h, l, d, ... are output repeatedly.

  Further, when the pixel signal output by shifting is restored to the original column by row unit, as shown in FIG. 10, the column of the pixel signal including the hatched noise is converted by row unit. The pixel signals including noise are distributed in the horizontal direction by shifting and outputting four columns to the left and right.

  As a result, pixel signals including noise are avoided from being concentrated and output in the same column, so that it is difficult to be recognized as vertical stripe noise.

  In the above description, the connection switching pattern of the switch 32c has been described with respect to the three types of patterns shown in FIGS. 6 to 8. However, the number of patterns for switching the switch 32c may be increased. There may be five types of connection switching patterns that shift four columns to the left, two columns to the left, zero columns, two columns to the right, and four columns to the right.

  Further, here, the example in which the number of columns shifted to the left and right in the connection switching pattern is symmetrical has been described, but it may be asymmetrical, for example, 4 columns on the left, 0 column, and 2 on the right. There may be three types of connection switching patterns that are asymmetrical to the left and right, such as columns.

  Furthermore, the number of connection switching patterns and the number of columns to be shifted may be selectively switched. That is, the first mode composed of three types of connection switching patterns that shift four columns to the left and right and the second mode composed of three types of connection switching patterns that shift ten columns to the left and right may be switched.

<3. Detailed configuration example of connection switching unit>
Next, a detailed configuration example of the connection switching unit 32 will be described with reference to FIG.

  For example, when the vertical transfer lines 34 are switched by shifting four columns to the left and right as described above, the vertical transfer lines 34 are divided into groups of four columns as shown in FIG. In FIG. 11, the inside of the connection switching unit 32 is enlarged, and some vertical transfer line groups 34G are drawn, and vertical transfer line groups 34G-11 to 34G-13 are displayed from the left.

  Each vertical transfer line group 34G is composed of four vertical transfer lines 34. In the figure, 1, 2, 3, and 4 are appended below the vertical transfer line group 34G, and each indicates a connection position of the vertical transfer lines 34 belonging to the vertical transfer line group 34G.

  In the vertical transfer group 34G-12, only the terminals 32a and 32b with respect to the vertical transfer line 34 with the third "3" from the left are representatively displayed as terminals 32a-x and 32b-x. It is written.

  In each vertical transfer group 34G-11, only the terminals 32a and 32b for the vertical transfer line 34 labeled "3" from the left are representatively displayed, and the terminals 32a- (x- 1) and 32b- (x-1).

  Further, in each vertical transfer group 34G-13, only the terminals 32a and 32b for the vertical transfer line 34 labeled "3" from the left are representatively displayed, and the terminal 32a- (x + 1). , 32b- (x + 1).

  That is, in FIG. 11, for each vertical transfer line group 34G, only the wiring of the terminals 32a and 32b in the third column from the left is representatively shown, and the other terminals 32a and 32b are not shown. Has been.

  In each vertical transfer line 34, the terminals corresponding to the terminals 32a and 32b in FIG. 2 are branched into three.

  The terminals 32a-x of the vertical transfer line group 34G-12 near the center in FIG. 11 are terminals 32a-x-1 and 34G-34 directed to the vertical transfer line group 34G-11 on the left in the figure. 12 is branched to a terminal 32a-x-2 toward the terminal 12 and a terminal 32a-x-3 toward the vertical transfer line group 34G-12 on the right side in the drawing.

  Similarly, the terminal 32b-x of the vertical transfer line group 34G-12 near the center in FIG. 11 is the terminal 32b-x-1, the vertical transfer line directed to the vertical transfer line group 34G-11 on the left in the figure. A branch is made to a terminal 32b-x-2 toward the group 34G-12 and a terminal 32b-x-3 toward the vertical transfer line group 34G-12 on the right side in the drawing.

  Similarly, for the vertical transfer line group 34G-11, the terminal 32a- (x-1) is branched into terminals 32a- (x-1) -1 to 32a- (x-1) -3, and the terminal 32b- ( x-1) is branched to terminals 32a- (x-1) -1 to 32a- (x-1) -3.

  Similarly, for the vertical transfer line group 34G-13, the terminal 32b- (x + 1) is branched into terminals 32b- (x + 1) -1 to 32b- (x + 1) -3, and the terminal 32b- (x + 1) is Branched to terminals 32b- (x + 1) -1 to 32b- (x + 1) -3.

  Further, the terminal 32a-x-1 of the vertical transfer line group 34G-12 is connected to the terminal 32b- (x-1) -3 of the vertical transfer line group 34G-11 via the switch SW11-3.

  The terminal 32a-x-2 of the vertical transfer line group 34G-12 is connected to the terminal 32b-x-2 of the vertical transfer line group 34G-12 via the switch SW12-2.

  Further, the terminal 32a-x-3 of the vertical transfer line group 34G-12 is connected to the terminal 32b- (x + 1) -1 of the vertical transfer line group 34G-13 via the switch SW13-1.

  Each of the switches SW11-1 to SW11-3, SW12-1 to SW12-3, SW13-1 to SW13-3 is, for example, a transfer composed of transistors Tr1 and Tr2, as shown in the lower left part of FIG. Consists of switches.

  With this configuration, the terminal 32a-x has been described with reference to FIGS. 6 to 8 because only one of the switches SW11-3, SW12-2, and SW13-1 is cyclically turned on. As described above, the vertical transfer line 34 and the AD converter 33a are switched so as to be shifted by four columns to the left and right.

  That is, when the switch SW11-3 is turned on and the switches SW12-2 and SW13-1 are turned off, the terminals 32a-x-1 of the vertical transfer line group 34G-12 are connected to the vertical transfer line group 34G-11. The terminal 32b- (x-1) -3 is electrically connected. Thereby, the terminals 32a-x in the third column from the left of the vertical transfer group 34G-12 are connected to the terminals 32b- (x-1) in the third column from the left of the vertical transfer group 34G-11. As a result, the terminals 32a-x of the vertical transfer line group 34G-12 are connected to the terminals 32b- (x-1) of the vertical transfer line group 34G-11, which are terminals adjacent to the left by four columns of the vertical transfer lines 34. Connected.

  When the switch SW12-2 is turned on and the switches SW11-3 and SW13-1 are turned off, the terminals 32a-x-1 of the vertical transfer line group 34G-12 are connected to the vertical transfer line group 34G-12. The terminal 32b-x-2 is electrically connected. Thereby, the terminals 32a-x in the third column from the left of the vertical transfer group 34G-12 are connected to the terminals 32b-x in the third column from the left of the vertical transfer group 34G-12. As a result, the terminals 32a-x of the vertical transfer line group 34G-12 are connected to the terminals 32b-x of the vertical transfer line group 34G-12, where the vertical transfer lines 34 are terminals in the same column.

  Further, when the switch SW13-1 is turned on and the switches SW11-3 and SW12-2 are turned off, the terminals 32a-x-1 of the vertical transfer line group 34G-12 are connected to the vertical transfer line group 34G-13. The terminal 32b- (x + 1) -1 is electrically connected. As a result, the terminal 32a-x in the third column from the left of the vertical transfer group 34G-12 is connected to the terminal 32b- (x + 1) in the third column from the left of the vertical transfer group 34G-13. As a result, the terminals 32a-x of the vertical transfer line group 34G-12 are connected to the terminals 32b- (x + 1) of the vertical transfer line group 34G-13, in which the vertical transfer lines 34 are terminals adjacent to the right by four columns. The

  The connection switching unit 32 controls the switches SW11-3, SW12-2, and SW13-1 as described above, and outputs the third column output from the left of the vertical transfer line group 34G-12 to the vertical transfer line 34. The output is switched between shifting to the left and right for four columns or outputting to the vertical transfer line 34 in the same column.

  The connection switching unit 32 controls the other vertical transfer lines 34 (not shown) of the vertical battle line group 34G-12 in the same manner, and also controls the other vertical transfer line groups 34G in the same manner. The terminal 32a, 32b of the vertical transfer line 34 is shifted to the left or right in units of four columns, respectively, or the output is switched to the vertical transfer line 34 of the same column.

  That is, with such control, the connection of the switch 32c-t (t is the column number of the vertical transfer line 34) in FIG. 2 is connected to the left terminal of the fourth column with reference to the terminal 32a-t in the vertical transfer line 34. 32b- (t-4), switching to the terminal 32b-t which is the same vertical transfer line, or the terminal 32b- (t + 4) adjacent to the right of the four columns is realized.

  2 and 11, the connection switching unit 32 controls the switch 32c using the terminal 32a as a reference to switch the connection destination terminal 32b. However, the terminal 32b is used as a reference. The switch 32c may be controlled to switch the connection destination terminal 32a.

<4. Configuration example of row end of connection switching unit>
Next, a configuration example of the column end portion of the connection switching unit 32 will be described with reference to FIGS. 12 to 14 are configuration examples of the left column end of the connection switching unit 32 in FIG. 2, for example.

  When the column AD circuit 33 including the same number of AD converters 33a as the vertical transfer lines 34 is configured, the connection switching unit 32 controls the switch 32c with the terminal 32a as a reference, and is adjacent to any of the four columns on the left and right. When the control is performed so as to switch to the terminal 32b of the shifted vertical transfer line 34, any of the four columns at the left and right column ends is in a state where there is no AD converter 33a to be connected.

  Therefore, in the connection switching unit 32 and the column AD circuit 33 according to the present disclosure, as illustrated in FIG. 12, a dummy AD converter 33a corresponding to the number of columns to be shifted is provided outside the normal vertical transfer line 34. The connection switching unit 32 is provided with terminals 32a and 32b corresponding to the dummy AD converter 33a.

  That is, as shown in FIG. 12, the column AD circuit 33 is provided with dummy AD converters 33a-d1 to 33a-d4 for four columns that are the same as the number of columns of the vertical transfer line 34 to be shifted. Yes. The connection switching unit 32 includes terminals 32a-d1 to 32a-d4 and 32b-d1 to 32b-d4 corresponding to the dummy AD converters 33a-d1 to 33a-d4. The terminals 32a-d1 to 32a-d4 are connected to the power supply potential VDD.

  When the switches 32c-1 to 32c-4 are not shifted and connected to the left and right, as shown in FIG. 12, the terminals 32a-d1 to 32a-d4 are respectively connected to the terminals 32b-d1 to 32b-d4. Connected to. In the case of FIG. 12, the AD converters 33a-d1 to 33a-d4 convert the power supply voltage VDD into a digital signal and output it.

  Further, when the switches 32c-1 to 32c-4 are connected by shifting to the left four columns, as shown in FIG. 13, the terminals 32a-1 to 32a-4 are respectively connected to the terminals 32b-d1 to 32b-d1. 32b-d4. At this time, the dummy AD converters 33a-d1 to 33a-d4 convert the pixel signals of the vertical transfer lines 34-1 to 34-4 into digital signals and output them. The AD converters 33a-1 to 33a-4 convert the pixel signals of the vertical transfer lines 34-5 to 34-8 into digital signals and output the digital signals.

  Further, when the switches 32c-1 to 32c-4 are shifted and connected in the four columns on the right side, as shown in FIG. 14, the terminals 32a-d1 to 32a are provided by the switches 32c-d1 to 32c-d4. -D4 is connected to terminals 32b-1 to 32b-4, respectively. Also, the terminals 32a-1 to 32a-4 are connected to the terminals 32b-5 to 32b-8, respectively. At this time, the AD converters 33a-1 to 33a-4 convert the power supply potential VDD into a digital signal and output it. The AD converters 33a-5 to 33a-8 convert the pixel signals of the vertical transfer lines 34-1 to 34-4 into digital signals and output the digital signals.

  Accordingly, the AD converters 33a-1 to 33a-4 of the column AD circuit 33 convert the power supply potential VDD into a digital signal once in the fixed switching process of the three-row cycle by the connection switching unit 32. It is stored in the memory 35. When the pixel signal stored in the memory 35 is read out by the reading unit 36, it is restored to the original column and read out, so that the pixel signal in which the power supply potential VDD is converted into a digital signal is substantially discarded.

  Since the dummy AD converters 33a-d1 to 33a-d4, terminals 32a-d1 to 32a-d4, and terminals 32b-d1 to 32b-d4 have a shift number of 4, the case of four is described. However, other numbers may be used depending on the number of shifts.

For example, in the case where the mode that is the three types of connection switching patterns with four columns on the left and right and the mode that is the three types of connection switching patterns with eight columns on the left and right are switched, the maximum number of eight columns. A dummy AD converter 33a and terminals 32a and 32b may be provided. Further, as described above, in order to disperse the pixel signals including noise to the left and right, the number of shifts is preferably a certain number of columns, for example, four or more columns. Further, when control is performed such that the number of shift columns (number of dummy columns) is switched in bit units, the number of shifts (number of dummy columns) is set to 2 ^k (k is a natural number), and k is a shift column. The number (number of dummy columns) may be switched.

  Although the configuration of the left column end portion of the column AD circuit 33 has been described above, the right column end portion is also provided with a configuration that is symmetrical and has the same function. In the above description, an example in which the dummy terminals 32a-d1 to 32a-d4 are connected to the power supply voltage VDD has been described. However, any other potential may be used because it may be a specific potential. It may be ground potential.

<5. Filter processing>
Next, processing of the filter 37 will be described with reference to FIGS. 15 to 20.

  For example, consider an image composed of 5 pixels (rows) × 5 pixels (columns) as shown in FIG. In FIG. 15, the pixel signal at the corresponding pixel position is shown as a decimal pixel value in the range of 0 to 255 in each square.

  For example, in the case where the connection switching unit 32 is not provided as shown in FIG. 3, the pixel value of the pixel signal that is the AD conversion result including noise corresponding to the AD converter 33a-8 is, for example, FIG. 15 is output as 255 in all rows of the second column, the pixel value of the other pixel positions is 10, and the vertical streak noise is visually recognized in the second column which is the center column of the image of FIG. It shall be.

  For an image as shown in FIG. 15, for example, for each pixel, for the central pixel in the range of 3 rows × 3 columns, using the pixel value of a total of 5 pixels including the central pixel and pixels adjacent vertically and horizontally. When the moving average filter is applied to each pixel in the range of 3 rows × 3 columns in the center of the image in FIG. 15, the pixel values are arranged as shown in FIG.

  Further, for the image as shown in FIG. 15, for example, for each pixel, for the central pixel in the range of 3 rows × 3 columns, a total of 5 pixels including the central pixel and the pixels adjacent vertically and horizontally are used. When the median filter is used for each pixel in the range of 3 rows × 3 columns in the center of FIG. 15, the pixel values are arranged as shown in FIG.

  In the range of 3 rows × 3 columns surrounded by the thick line in the center of FIGS. 16 and 17, the value of the second column of the center column is 157 or 255 regardless of which filter processing is performed. It is a large value compared with other pixel values 59 or 10, and as a result, it is shown that the vertical stripe noise is still easily recognized.

  On the other hand, when the pixel signal in FIG. 15 is AD-converted by shifting the pixel signal by two columns left and right for each row, that is, by the connection switching unit 32, the vertical transfer line 34 When the switching process without shift, with two column shifts on the left and with two column shifts on the right is repeated in a three-row cycle, an image in which pixel signals having pixel values as shown in FIG. 18 are arranged is obtained.

  That is, in FIG. 18, the pixel signal of pixel value 255 is the second column of the center column in the uppermost row, the 0th column in the first row, the fourth column in the second row, the fourth column in the third row, the fourth column in the third row, In the row, the 0th column is set, and the other pixel signals are set to 10.

  For the image signal of 5 rows × 5 columns composed of the pixel signals of FIG. 18, for example, for each pixel, the center pixel in the range of 3 rows × 3 columns and the pixels adjacent in the vertical and horizontal directions When a moving average filter is used using a total of five pixels, the pixel signal is arranged with pixel values as shown in FIG.

  That is, as shown in FIG. 19, in the range of 3 rows × 3 columns surrounded by the central thick line, when moving average filter processing is performed, only 2 out of 9 pixels are 10 and other pixels Becomes 59, and the influence of vertical streak noise can be reduced.

  Further, for the image signal of 5 rows × 5 columns composed of the pixel signals of FIG. 18, for example, for each pixel, the central pixel in the range of 3 rows × 3 columns is adjacent to the central pixel in the vertical and horizontal directions When the median filter is used using a total of 5 pixels, the pixel signal arrangement as shown in FIG. 20 is obtained.

  That is, as shown in FIG. 20, in the range of 3 rows × 3 columns surrounded by the central thick line, when median filtering is performed, all 9 pixels become 10 and the effect of vertical stripe noise is almost complete. Can be eliminated.

  That is, as described above, the connection switching unit 32 switches between the vertical transfer line 34 and each AD converter 33a of the column AD circuit 33 so as to shift by a predetermined number of columns by a predetermined number of rows. When the output pixel signal is restored, it is possible to recognize which column of pixel signal is converted into a digital signal by which AD converter 33a with respect to the vertical transfer line 34 in units of rows. . Therefore, the influence of vertical stripe noise can be further reduced by an appropriate filter process based on the information on the number of shift columns in units of rows by the connection switching unit 32.

  In addition, although the example using the moving average filter and the median filter has been described above, the filter may be other filters.

<6. Imaging processing>
Next, imaging processing by the imaging device 11 of FIG. 1 will be described with reference to the flowchart of FIG.

  In step S31, the connection switching unit 32 initializes a counter T for identifying a connection switching pattern to 0. The connection switching pattern here is a connection switching pattern of the switch 32c in FIGS.

  In the description of the flowchart of FIG. 21, the counter T = 0 is a connection switching pattern in which the vertical transfer line 34 of FIG. 6 is connected without shifting, and the counter T = 1 is the left direction shown in FIG. It is assumed that the connection switching pattern is shifted by 4 columns and the counter T = 2 is the connection switching pattern shifted by 4 columns in the right direction shown in FIG.

  In step S32, the connection switching unit 32 switches the switch 32c so as to connect the terminals 32a and 32b with the connection switching pattern corresponding to the counter T.

  In step S <b> 33, a pixel signal is transferred from each pixel 51 in a predetermined row of the pixel array 31 to the column AD circuit 33. That is, each pixel signal of the vertical transfer line 34 is output to each AD converter 33 a of the column AD circuit 33 in a state where the connection of the switch 32 c is switched to the connection switching pattern identified by the counter T.

  In step S34, each AD converter 33a of the column AD circuit 33 converts the pixel signal supplied via the connection switching unit 32 into a digital signal, and associates it with information for identifying each column in units of rows. It is stored in the memory 35.

  In step S35, the connection switching unit 32 determines whether or not the pixel signals of all rows have been transferred. If the pixel signals of all rows have not been transferred, the process proceeds to step S36.

  In step S36, the connection switching unit 32 determines whether or not the counter T is 2, which is the maximum value. In step S36, when the counter T is not 2, which is the maximum value, the process proceeds to step S37.

  In step S37, the connection switching unit 32 increments the counter T by 1, and the process returns to step S32.

  If it is determined in step S36 that the counter T is 2, the process returns to step S31.

  That is, when the counter T is 0 to 2, the processing of steps S32 to S37 is repeated, so that the connection switching patterns of FIGS. And repeated until the pixel signals of all rows are transferred.

  If it is determined in step S35 that the pixel signals of all rows have been transferred, the process proceeds to step S38.

  In step S <b> 38, the readout unit 36 reads out the pixel signal stored in the memory 35 so as to restore it according to the connection switching pattern, and outputs it to the filter 37.

  That is, the pixel signal is stored in the memory 35, for example, as shown in FIG. 9, by switching the connection switching pattern by the counter T. Therefore, the reading unit 36 is shown in FIG. The pixel signal is read out so that the original column of the correct pixel signal is restored.

  In step S39, as described with reference to FIGS. 15 to 20, the filter 37 performs a filtering process such as a moving average filter or a median filter, and outputs the result.

  Since the connection switching pattern is switched in units of rows by the above processing, the AD converter 33a of the column AD circuit 33 that performs digital conversion including noise is provided at a position corresponding to one of the vertical transfer lines 34. Even in such a case, as described with reference to FIG. 10, a pixel signal including noise can be converted into a digital signal by shifting the pixel signal including noise to a specific column in units of rows. The signals can be dispersed so as not to be aligned with a specific column, and the visibility of vertical stripe noise can be reduced by reducing the correlation in the column direction.

  In addition, pixel signals converted into noise-containing digital signals are distributed and read out in different columns for each row in a fixed pattern at a predetermined row cycle, so that an appropriate corresponding to the fixed pattern With this filtering process, it is possible to reduce the correlation of pixels including noise in the vertical direction, and it is possible to reduce the visibility of vertical stripe noise.

  Furthermore, when realizing the above-described processing, it is not necessary to switch the connection switching pattern at random by changing the connection switching pattern by a different number of shifts for each row with a fixed pattern in a predetermined row cycle. Therefore, it is not necessary to add a random number generation circuit or the like, and the visibility of vertical stripe noise can be reduced without increasing the chip size and cost.

  In the above description, an example of switching to a different connection switching pattern for each row has been described, but switching to a different connection switching pattern may be performed for each predetermined number of rows.

<7. Application example to electronic equipment>
The above-described imaging device can be applied to various electronic devices such as an imaging device such as a digital still camera or a digital video camera, a mobile phone having an imaging function, or other devices having an imaging function.

  FIG. 22 is a block diagram illustrating a configuration example of an imaging device as an electronic apparatus to which the present technology is applied.

  An imaging apparatus 201 shown in FIG. 22 includes an optical system 202, a shutter device 203, a solid-state imaging device 204, a drive circuit 205, a signal processing circuit 206, a monitor 207, and a memory 208, and displays still images and moving images. Imaging is possible.

  The optical system 202 includes one or more lenses, guides light (incident light) from a subject to the solid-state image sensor 204, and forms an image on the light receiving surface of the solid-state image sensor 204.

  The shutter device 203 is disposed between the optical system 202 and the solid-state image sensor 204, and controls the light irradiation period and the light-shielding period to the solid-state image sensor 204 according to the control of the drive circuit 1005.

  The solid-state image sensor 204 is configured by a package including the above-described solid-state image sensor. The solid-state imaging device 204 accumulates signal charges for a certain period in accordance with light imaged on the light receiving surface via the optical system 202 and the shutter device 203. The signal charge accumulated in the solid-state image sensor 204 is transferred according to a drive signal (timing signal) supplied from the drive circuit 205.

  The drive circuit 205 outputs a drive signal for controlling the transfer operation of the solid-state image sensor 204 and the shutter operation of the shutter device 203 to drive the solid-state image sensor 204 and the shutter device 203.

  The signal processing circuit 206 performs various types of signal processing on the signal charge output from the solid-state imaging device 204. An image (image data) obtained by the signal processing by the signal processing circuit 206 is supplied to the monitor 207 and displayed, or supplied to the memory 208 and stored (recorded).

Also in the imaging apparatus 201 configured as described above, the configuration of the entire apparatus can be reduced in size by applying the imaging element 11 illustrated in FIG. 1 of the present disclosure instead of the solid-state imaging element 204 described above. It becomes possible.
<8. Example of using image sensor>

  FIG. 23 is a diagram illustrating a usage example in which the imaging device 201 of FIG. 22 described above is used.

  The imaging device described above can be used in various cases for sensing light such as visible light, infrared light, ultraviolet light, and X-ray as follows.

・ Devices for taking images for viewing, such as digital cameras and mobile devices with camera functions ・ For safe driving such as automatic stop and recognition of the driver's condition, Devices used for traffic, such as in-vehicle sensors that capture the back, surroundings, and interiors of vehicles, surveillance cameras that monitor traveling vehicles and roads, and ranging sensors that measure distances between vehicles, etc. Equipment used for home appliances such as TVs, refrigerators, air conditioners, etc. to take pictures and operate the equipment according to the gestures ・ Endoscopes, equipment that performs blood vessel photography by receiving infrared light, etc. Equipment used for medical and health care ・ Security equipment such as security surveillance cameras and personal authentication cameras ・ Skin measuring instrument for photographing skin and scalp photography Such as a microscope to do beauty Equipment used for sports-Equipment used for sports such as action cameras and wearable cameras for sports applications-Used for agriculture such as cameras for monitoring the condition of fields and crops apparatus

<9. Application example>
The technology according to the present disclosure (present technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device that is mounted on any type of mobile body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, and a robot. May be.

  FIG. 24 is a block diagram illustrating a schematic configuration example of a vehicle control system that is an example of a mobile control system to which the technology according to the present disclosure can be applied.

  The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in FIG. 24, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. As a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are illustrated.

  The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 includes a driving force generator for generating a driving force of a vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism that adjusts and a braking device that generates a braking force of the vehicle.

  The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a blinker, or a fog lamp. In this case, the body control unit 12020 can be input with radio waves transmitted from a portable device that substitutes for a key or signals from various switches. The body system control unit 12020 receives input of these radio waves or signals, and controls a door lock device, a power window device, a lamp, and the like of the vehicle.

  The vehicle outside information detection unit 12030 detects information outside the vehicle on which the vehicle control system 12000 is mounted. For example, the imaging unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The vehicle outside information detection unit 12030 may perform an object detection process or a distance detection process such as a person, a car, an obstacle, a sign, or a character on a road surface based on the received image.

  The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of received light. The imaging unit 12031 can output an electrical signal as an image, or can output it as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared rays.

  The vehicle interior information detection unit 12040 detects vehicle interior information. For example, a driver state detection unit 12041 that detects a driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the vehicle interior information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated or it may be determined whether the driver is asleep.

  The microcomputer 12051 calculates a control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside / outside the vehicle acquired by the vehicle outside information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit A control command can be output to 12010. For example, the microcomputer 12051 realizes an ADAS (Advanced Driver Assistance System) function including vehicle collision avoidance or impact mitigation, vehicle-following travel based on inter-vehicle distance, vehicle speed maintenance travel, vehicle collision warning, or vehicle lane departure warning. It is possible to perform cooperative control for the purpose.

  Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform cooperative control for the purpose of automatic driving that autonomously travels without depending on the operation.

  Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on information outside the vehicle acquired by the vehicle outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamp according to the position of the preceding vehicle or the oncoming vehicle detected by the outside information detection unit 12030, and performs cooperative control for the purpose of anti-glare, such as switching from a high beam to a low beam. It can be carried out.

  The sound image output unit 12052 transmits an output signal of at least one of sound and image to an output device capable of visually or audibly notifying information to a vehicle occupant or the outside of the vehicle. In the example of FIG. 24, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

  FIG. 25 is a diagram illustrating an example of the installation position of the imaging unit 12031.

  In FIG. 25, the vehicle 12100 includes imaging units 12101, 12102, 12103, 12104, and 12105 as the imaging unit 12031.

  The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as a front nose, a side mirror, a rear bumper, a back door, and an upper portion of a windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided in the front nose and the imaging unit 12105 provided in the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The imaging units 12102 and 12103 provided in the side mirror mainly acquire an image of the side of the vehicle 12100. The imaging unit 12104 provided in the rear bumper or the back door mainly acquires an image behind the vehicle 12100. The forward images acquired by the imaging units 12101 and 12105 are mainly used for detecting a preceding vehicle or a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

  FIG. 25 shows an example of the shooting range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively, and the imaging range 12114 The imaging range of the imaging part 12104 provided in the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, an overhead image when the vehicle 12100 is viewed from above is obtained.

  At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

  For example, the microcomputer 12051, based on the distance information obtained from the imaging units 12101 to 12104, the distance to each three-dimensional object in the imaging range 12111 to 12114 and the temporal change in this distance (relative speed with respect to the vehicle 12100). In particular, it is possible to extract, as a preceding vehicle, a three-dimensional object that travels at a predetermined speed (for example, 0 km / h or more) in the same direction as the vehicle 12100, particularly the closest three-dimensional object on the traveling path of the vehicle 12100. it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in advance before the preceding vehicle, and can perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. Thus, cooperative control for the purpose of autonomous driving or the like autonomously traveling without depending on the operation of the driver can be performed.

  For example, the microcomputer 12051 converts the three-dimensional object data related to the three-dimensional object to other three-dimensional objects such as a two-wheeled vehicle, a normal vehicle, a large vehicle, a pedestrian, and a utility pole based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 is connected via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration or avoidance steering via the drive system control unit 12010, driving assistance for collision avoidance can be performed.

  At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the captured images of the imaging units 12101 to 12104. Such pedestrian recognition is, for example, whether or not the user is a pedestrian by performing a pattern matching process on a sequence of feature points indicating the outline of an object and a procedure for extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras. It is carried out by the procedure for determining. When the microcomputer 12051 determines that there is a pedestrian in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 has a rectangular contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to be superimposed and displayed. Moreover, the audio | voice image output part 12052 may control the display part 12062 so that the icon etc. which show a pedestrian may be displayed on a desired position.

  Heretofore, an example of a vehicle control system to which the technology according to the present disclosure can be applied has been described. The technology according to the present disclosure can be applied to a block to which the present application is applied, for example, the imaging unit 12031 among the configurations described above. Specifically, for example, the imaging device 1 in FIG. 1 can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, it is possible to reduce the correlation in the column direction of noise generated by the column AD circuit, and to make it difficult to visually recognize vertical stripe noise.

In addition, this indication can also take the following structures.
<1> a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array;
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An image pickup device comprising: a connection switching unit that shifts a connection between the vertical transfer line and the AD converter by a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line. .
<2> The pixel signal AD-converted by the AD converter in a state where the connection switching unit is shifted by the predetermined number of columns by the connection switching unit, before the connection is switched. The imaging device according to <1>, further including a readout unit that reads out the pixel signal of the column.
<3> The storage switching unit further includes a storage unit that stores the pixel signal that is AD-converted by the AD converter in a state where the connection is switched by shifting the predetermined number of columns,
The readout unit stores the pixel signal AD-converted by the AD converter in a state where the connection switching unit shifts the predetermined number of columns and is switched by the connection switching unit. The image sensor according to <2>, wherein the pixel signal is read out as a pixel signal of the original column before the connection is switched.
<4> The imaging element according to any one of <1> to <3>, wherein the predetermined number of columns changes each time the pixel signal is transferred in units of rows by the vertical transfer line.
<5> The imaging device according to <4>, wherein the predetermined number of columns is fixedly changed with a predetermined number of times that the pixel signal is transferred in units of rows by the vertical transfer line as a cycle.
<6> When the pixel signal is transferred in units of rows by the vertical transfer line, the connection switching unit connects the vertical transfer line and the AD converter by the predetermined number of columns. When the pixel signal is transferred next after shifting in the direction and switching the connection, the connection is switched by shifting in the second direction by the predetermined number of columns. The imaging device according to <5>, in which the connection is repeated without being shifted when transferred.
<7> The imaging device according to <4>, wherein the predetermined number of columns is 2 ^k (k is a natural number).
<8> The imaging device according to <2>, further including a filter unit that performs a filtering process on the pixel signal restored and read out as the pixel signal before the connection is switched by the reading unit.
<9> The imaging device according to <8>, wherein the filter unit is a moving average filter or a median filter.
<10> The connection switching unit
An output terminal of the vertical transfer line in the column unit;
An input terminal of the AD converter in units of columns;
A connection changeover switch for switching the connection between the output terminal and the input terminal;
The image sensor according to any one of <1> to <9>, wherein the connection switch is controlled to switch the connection between the output terminal and the input terminal each time a pixel signal is output in units of rows. .
<11> a first conduction switch for switching on or off the connection between the output terminal and the input terminal of the column shifted in the first direction by the predetermined number of columns;
A second conduction switch for switching on or off the connection between the output terminal and the input terminal of the column shifted in the second direction by the predetermined number of columns;
A third conduction switch for switching on or off the connection between the output terminal and the input terminal in the same column;
The connection switching unit controls on / off of the first to third conduction switches to control the connection switching switch each time a pixel signal is output in units of rows, so that the output terminal And switching the connection with the input terminal. <10>.
<12> The column AD circuit includes:
Including the same number of dummy AD converters as the number of columns to be shifted at the column end,
The connection switching unit
Dummy input terminals of the dummy AD converters in the number of columns to be shifted, the same number as the number of columns to be shifted;
Including the same number of dummy connection changeover switches as the number of columns to be shifted, which switches the connection between the vertical transfer line at the column end and the dummy input terminal,
When the connection between the vertical transfer line and the AD converter is switched by shifting the predetermined number of columns, the dummy connection changeover switch is controlled to control the vertical transfer line at the column end and the column. The imaging according to any one of <1> to <11>, wherein a dummy input terminal at an end is connected, and a pixel signal transferred through the vertical transfer line at the column end is AD-converted by the dummy AD converter. element.
<13> A dummy output terminal of a dummy vertical transfer line having a power supply potential or a ground potential is further included at the end of the column,
When the connection between the vertical transfer line and the AD converter is switched to connect in the same column, the connection switching unit controls the dummy connection switch, the dummy output terminal, The imaging device according to <12>, which is connected to a dummy input terminal.
<14> a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array;
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A control method of an image sensor including a column AD converter having a plurality of AD converters that perform AD (Analog to Digital) conversion of the pixel signal for each column,
A method for controlling an image sensor, comprising: switching a connection between the vertical transfer line and the AD converter by shifting a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line .
<15> a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array;
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An imaging apparatus comprising: a connection switching unit that switches the connection between the vertical transfer line and the AD converter by a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line. .
<16> a pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array;
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An electronic device comprising: a connection switching unit that switches a connection between the vertical transfer line and the AD converter by shifting a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line .

  11 imaging device, 31 pixel array, 32 imaging device, 32a, 32a-1 to 32a-16, 32a-d1 to 32a-d4, 32b, 32b-1 to 32b-16, 32b-d1 to 32b-d4 terminal, 32c , 32c-1 to 32c-16 switch, 33 column AD, 33a, 33a-1 to 33a-16 AD converter, 34, 34-1 to 34-16 vertical transfer line, 35 memory, 36 reading unit, 37 filter

Claims (16)

A pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array; and
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An image pickup device comprising: a connection switching unit that shifts a connection between the vertical transfer line and the AD converter by a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line. . The pixel in the original column before the connection is switched with the pixel signal AD-converted by the AD converter in a state where the connection is shifted by the connection switching unit and the connection is switched. The imaging device according to claim 1, further comprising a reading unit that reads out as a signal. In the state where the connection switching unit is shifted by the predetermined number of columns and the connection is switched, the storage unit further includes a storage unit that stores the pixel signal subjected to AD conversion by the AD converter,
The readout unit stores the pixel signal AD-converted by the AD converter in a state where the connection switching unit shifts the predetermined number of columns and is switched by the connection switching unit. The image sensor according to claim 2, wherein the pixel signal is read out as a pixel signal of the original column before the connection is switched. The imaging device according to claim 1, wherein the predetermined number of columns changes each time the pixel signal is transferred in units of rows by the vertical transfer line. The imaging device according to claim 4, wherein the predetermined number of columns is fixedly changed with a predetermined number of times that the pixel signal is transferred in units of rows by the vertical transfer line as a cycle. The connection switching unit shifts the connection between the vertical transfer line and the AD converter in the first direction by the predetermined number of columns when the pixel signal is transferred in units of rows by the vertical transfer line. Then, when the pixel signal is transferred next after switching the connection, the connection is switched by shifting in the second direction by the predetermined number of columns, and then the pixel signal is transferred next. The imaging device according to claim 5, wherein the connection is repeated without shifting. The imaging device according to claim 4, wherein the predetermined number of columns is 2 ^k (k is a natural number). The imaging device according to claim 2, further comprising a filter unit that performs a filtering process on the pixel signal restored and read out as a pixel signal before the connection is switched by the reading unit. The image sensor according to claim 8, wherein the filter unit is a moving average filter or a median filter. The connection switching unit
An output terminal of the vertical transfer line in the column unit;
An input terminal of the AD converter in units of columns;
A connection changeover switch for switching the connection between the output terminal and the input terminal;
The imaging device according to claim 1, wherein each time a pixel signal is output in units of rows, the connection selector switch is controlled to switch a connection between the output terminal and the input terminal. A first conduction switch for switching on or off connection between the output terminal and the input terminal of the column shifted in the first direction by the predetermined number of columns;
A second conduction switch for switching on or off the connection between the output terminal and the input terminal of the column shifted in the second direction by the predetermined number of columns;
A third conduction switch for switching on or off the connection between the output terminal and the input terminal in the same column;
The connection switching unit controls on / off of the first to third conduction switches to control the connection switching switch each time a pixel signal is output in units of rows, so that the output terminal The imaging device according to claim 10, wherein the connection with the input terminal is switched. The column AD circuit is:
Including the same number of dummy AD converters as the number of columns to be shifted at the column end,
The connection switching unit
Dummy input terminals of the dummy AD converters in the number of columns to be shifted, the same number as the number of columns to be shifted;
Including the same number of dummy connection changeover switches as the number of columns to be shifted, which switches the connection between the vertical transfer line at the column end and the dummy input terminal,
When the connection between the vertical transfer line and the AD converter is switched by shifting the predetermined number of columns, the dummy connection changeover switch is controlled to control the vertical transfer line at the column end and the column. The image sensor according to claim 1, wherein a dummy input terminal at an end is connected, and the pixel signal transferred through the vertical transfer line at the column end is AD converted by the dummy AD converter. The column end further includes a dummy output terminal of a dummy vertical transfer line of a power supply potential or a ground potential,
When the connection between the vertical transfer line and the AD converter is switched to connect in the same column, the connection switching unit controls the dummy connection switch, the dummy output terminal, The imaging device according to claim 12, wherein the imaging device is connected to a dummy input terminal. A pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array; and
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A control method of an image sensor including a column AD converter having a plurality of AD converters that perform AD (Analog to Digital) conversion of the pixel signal for each column,
A method for controlling an image sensor, comprising: switching a connection between the vertical transfer line and the AD converter by shifting a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line . A pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array; and
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An imaging apparatus comprising: a connection switching unit that switches the connection between the vertical transfer line and the AD converter by a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line. . A pixel array that outputs a pixel signal corresponding to the amount of incident light in units of pixels arranged in an array; and
A plurality of vertical transfer lines for transferring the pixel signal in units of rows for each column;
A column AD converter having a plurality of AD converters for AD (Analog to Digital) conversion of the pixel signal for each column;
An electronic device comprising: a connection switching unit that switches a connection between the vertical transfer line and the AD converter by shifting a predetermined number of columns each time the pixel signal is transferred in units of rows by the vertical transfer line .

Images