JPH1155571A - Image processing unit and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain the image processing unit by which 2-dimension filtering is realized by a simple pixel structure while extracting an image. SOLUTION: The unit is made up of a 2-dimension pixel array 2 where pluralities of unit pixel circuits are arranged in a 2-dimension array, a pixel data memory array 4 where number of memory cells 3 is arranged by number of columns of the pixel arrays 2 and (n) rows (n is a natural number) of memory cells 3 are arranged vertically, a data transfer means that transfers pixel data from a selected unit pixel circuit in the unit of rows to each memory cell of a prescribed row of the pixel data memory array 4 to store pixel data, and an output means that selects m-columns (m is a natural number) of the memory cells for output purpose from the pixel data memory array 4 and extracts in parallel each output from selected n×m memory cells.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and a pixel processing method, and more particularly to an image processing apparatus and a pixel processing method capable of extracting an image projected on an image sensor while executing a filtering process using a two-dimensional kernel. The present invention relates to a light receiving element array and a configuration and a driving method of the array driving circuit.

[0002]

2. Description of the Related Art FIG. 17 is a block diagram showing a conventional image processing apparatus described in Japanese Patent Application No. 8-350160, showing a light receiving element circuit array and its peripheral circuits. Here, the filtering range (n × m pixels) is 3 ×
The case of three pixels is shown. The pixel array is 8
× 8 pixels are displayed. In the figure, 94 is a unit pixel circuit, 95 is a vertical scanning circuit for selecting n (here, 3) rows, and 96 is n (here, 3) for selecting m (here, 3) columns. 3) horizontal scanning circuits,
Two kernel bits are assigned to each column so that the first bit has a positive kernel value and the second bit has a negative kernel value for the selected pixel. 97 indicates the sum of the output currents output from pixels having a positive value in the kernel value and the output currents output from pixels having a negative value. This is a difference circuit which takes in by division and takes the difference between them.

The structure of the unit pixel circuit 94 is, for example, as shown in FIG. In FIG. 18, when the charge of the pixel data storage photodiode 98 is released due to light incidence, the conductance of the read transistor 99 changes, and the output is output when there is an input from both the column selection terminal 100 and the row selection terminal 101. Pixel data is extracted from the terminal 102 as a current.

Next, the filtering operation will be described with reference to FIG. The vertical scanning circuit 95 selects three rows to be filtered. _According to the value of 0 or 1 stored in the nine kernel bits K _{00+ to} K ₂₂₊ stored in the horizontal scanning circuit 96, the selected 3 × 3
, Pixel data is taken out from an arbitrary portion within the range, and summed and sent to the difference circuit 97. This corresponds to the positive weight part of the kernel. Next, the entire horizontal scanning circuit 96 is
After the bit shift, an arbitrary value within the selected 3 × 3 range is selected in accordance with the value of 0 or 1 stored in the nine kernel bits K _{00− to} K ₂₂₋ stored in the horizontal scanning circuit 96. Pixel data is extracted from the location, summed, and sent to the difference circuit 97. This corresponds to the negative weight part of the kernel. The difference circuit 97 extracts the positive weight portion and the negative weight portion of these kernels in a time-division manner and outputs the difference between them. After the output, the horizontal scanning circuit 96 is further shifted by one bit. By repeating this operation, the filtering range shifts one by one in the horizontal direction.
In this way, the image projected on the sensor can be extracted while performing two-dimensional filtering with kernel values of +1, 0, and -1.

FIG. 19 is a ISSCC Proceedings (199
4 ISSCC (Int. Solid-State)
Circuits Conf. ) Digest of
Technical Papers) pp. 228-229, “An Amplified MOS Im
ager Suited for Image Pro
sessing "(Masayuki Sugawar)
FIG. 7A is a configuration diagram showing another conventional light receiving element circuit array described in a). In the figure, 103 is a unit pixel circuit, 1
Reference numeral 04 denotes a pixel array in which unit pixel circuits are arranged, and reference numeral 105 denotes a three-column analog memory array having the same width as the pixel array 104. Reference numerals 106, 107, and 108 denote three independent clock signals φ1, φ2, and φ3, respectively, which are used to transfer pixel data of three consecutive rows of pixels to three columns of the memory array. 109, 110, 111
Is an output of each memory array, and a simple filtering in the vertical direction is realized by performing an operation between the outputs.

[0006]

In the conventional two-dimensional filtering light receiving element circuit array configured as shown in FIG. 17, it is necessary to arrange the same number of wires as the kernel size between each unit pixel circuit. Since the pixel structure becomes complicated, miniaturization of the pixel has been prevented. Further, only +1, 0, -1 could be taken as the weight, and the shape of the kernel was fixed.

Further, in a structure as shown in FIG. 19, in which pixel data is transferred to an analog memory and an operation is performed on the data in the memory, pixel data of pixels adjacent in the vertical direction are simply taken out simultaneously. The operations that can be performed are limited to one-dimensional filtering for three pixels that are consecutive in the vertical direction.

The present invention has been made to solve such a problem. In a light receiving element circuit array, an inter-pixel operation performed while extracting an image pattern can be performed by a two-dimensional kernel that can be programmed from the outside. It is an object of the present invention to provide an image processing apparatus and an image processing method having a simple pixel structure with multivalued weights and variable sizes.

[0009]

According to a first aspect of the present invention, there is provided an image processing apparatus comprising a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling an output of the element, which are arranged in a two-dimensional array. Memory cells having a two-dimensional pixel array, a pixel data memory, and a control circuit for controlling input / output of the memory are arranged in the number of columns of the pixel array in the horizontal direction and n rows (n: natural number) in the vertical direction. A) a pixel data memory array arranged side by side, a pixel array vertical scanning circuit for selecting a row in the two-dimensional pixel array, and a memory array vertical scanning circuit for selecting a row in the pixel data memory array in synchronization with the pixel array vertical scanning circuit. Pixel data from a plurality of unit pixel circuits in a row selected by the pixel array vertical scanning circuit by the memory array vertical scanning circuit in row units. Data transfer means for transferring the pixel data to the plurality of memory cells in the selected row and storing the pixel data in the pixel data memory; and memory cells for outputting from the pixel data memory array in m columns (m: natural number) It has a horizontal scanning circuit to select, and has output means for taking out respective outputs from the selected n × m memory cells in parallel.

In the image processing apparatus according to a second configuration of the present invention, each of the memory cells selects m (m: natural number, m ≧ 2) output terminals and the output terminal as an output destination of each memory cell. The output means connects the m output terminals to each other between the memory cells in the same row arranged in the horizontal direction to form n × m output lines, and The m selection terminals are connected to each other between the memory cells in the same column arranged in a row, and parallel outputs corresponding to the positions of the selected m columns of memory cells are taken out from the n × m output lines. It is.

According to a third aspect of the present invention, in the image processing apparatus according to the third aspect, the horizontal scanning circuit is configured so that the number X of memory cells in a horizontal direction is equal to X.
It has X input terminals corresponding to (X: natural number), and has a logic circuit for selecting m columns of the pixel data memory array in accordance with a variable access pattern with the position where the input was made as a reference point. is there.

An image processing apparatus according to a fourth configuration of the present invention includes an output circuit that weights each of the n × m outputs from the output means and takes the sum of the weighted outputs. It is.

An image processing apparatus according to a fifth aspect of the present invention is the image processing apparatus, wherein the output circuit includes means for collecting outputs to which the same weight is given and adding these outputs before weighting.

According to a sixth aspect of the present invention, in the image processing apparatus, the output circuit outputs one output from the output unit.
a weight selection unit composed of k (k: natural number) switches and a 1-bit storage device connected to each switch, and outputs the output from the output means to k weight setting units by the weight selection unit It is something to sort.

An image processing apparatus according to a seventh configuration of the present invention has a structure in which a storage device connected to each switch can be cyclically shifted in a vertical direction in a corresponding memory cell.

In an image processing apparatus according to an eighth aspect of the present invention, the pixel array vertical scanning circuit selects a row of the pixel array according to a variable access pattern, using an arbitrary row in the vertical direction of the pixel array as a reference point. Logic circuit that performs

According to a ninth aspect of the present invention, in the image processing apparatus according to the ninth aspect, the pixel array vertical scanning circuit has a structure in which shift registers are overlapped in two rows, each row can be shifted independently, and one row is connected to one side. It has a structure that allows you to copy a value to a series.

An image processing apparatus according to a tenth aspect of the present invention divides the pixel data memory array into two parts, a memory group corresponding to an even column and a memory group corresponding to an odd column of a two-dimensional pixel array. A horizontal scanning circuit is provided for each memory group.

In an image processing apparatus according to an eleventh aspect of the present invention, in the unit pixel circuit, wiring is divided between a power supply for charging a photoelectric conversion element and a power supply for a control circuit for controlling an output of the photoelectric conversion element. Things.

An image processing method according to the present invention is directed to a two-dimensional pixel array in which a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling the output of the element are arranged in a two-dimensional array. A pixel data memory array is provided in which a plurality of memory cells having a data memory and a control circuit for controlling input / output of the memory are arranged in a two-dimensional array,
Selecting a plurality of unit pixel circuits arranged in a row direction in the pixel array, and transferring pixel data to each memory cell in a predetermined row of the pixel data memory array in row units;
The pixel data is stored in the pixel data memory, and a plurality of columns of memory cells for outputting from the pixel data memory array are selected, and respective outputs from the selected memory cells are taken out in parallel. It is.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. Hereinafter, Embodiment 1 of the present invention will be described. FIG. 1 is a configuration diagram showing an image processing apparatus according to Embodiment 1 of the present invention, and shows a light receiving element circuit array (pixel array), a memory array, and peripheral circuits thereof. Here, a case of 4 × 4 is shown as the number of pixels n × m to be accessed at the same time, and an 8 × 8 pixel array is shown (actually, for example, 256 × 256, which corresponds to a part thereof). Conceivable). In the figure, 1 is a unit pixel circuit, 2 is a two-dimensional pixel array in which a plurality of unit pixel circuits 1 are arranged in a two-dimensional array, 3 is a unit memory cell, and 4 is a memory in which a plurality of unit memory cells 3 are arranged. An array (pixel data memory array) and a plurality of unit memory cells 3
Are arranged in the same number as the columns of the pixel array 2 in the horizontal direction, and are arranged in n rows in the vertical direction, here, four rows. 5 is a pixel array vertical scanning circuit for selecting a row to be accessed in the pixel array 2, 6 is a memory array vertical scanning circuit for selecting a row of the memory array 4, and 7 is pixel data for transferring pixel data to the memory array 4. This is a transfer line. Reference numeral 8 denotes a pixel reset shift register that selects a row for resetting the unit pixel circuit 1. In the present embodiment, a memory array 4 in which unit memory cells 3 in the same number of rows as the size n in the vertical direction of the kernel are prepared, and the pixel data of the row of the pixel array 2 to be filtered is stored in the memory array 4.
Transfer to The transfer is controlled using the pixel array vertical scanning circuit 5 and the memory array vertical scanning circuit 6, and is performed line by line through the pixel data transfer line 7. The kernel size x-direction setting logic 9 and the x-direction multiplexer 10 specify the x-coordinate range to be filtered on the memory array 4 when the horizontal direction is the x-axis and the vertical direction is the y-axis. Data of the desired n × m pixel data on the memory array is output as a parallel output 11. A weighting block 12 weights each of the n × m outputs, and sums the outputs to obtain a filtered output 13.
Output as

FIG. 2 shows an example of the structure of the unit pixel circuit 1. In the figure, a photodiode 1 for storing pixel data
4 is input to the gate terminal of the amplification MOS transistor 15. The output current from the MOS transistor 15 is controlled by a row selection terminal
Output terminal 1 of unit pixel circuit 1 through transistor 17
8 is output. The reset terminal 20 controls the reset MOS transistor 19 of the photodiode 14. Reference numeral 21 denotes a pixel unit power supply. The row selection terminal 16 is shown in FIG.
Reset terminal 2 by the pixel array vertical scanning circuit 5 of FIG.
0 is controlled by the pixel reset shift register 8 of FIG.

Next, the operation of the unit pixel circuit 1 shown in FIG. 2 will be described. First, the photodiode 14 is reset to the potential of the power supply 21 through the reset MOS transistor 19. Photodiode 1 by light incidence
When the charge is accumulated in 4, the conductance of the pixel data amplification MOS transistor 15 changes. Thereby, the output of the photodiode 14 is amplified, and the S / N ratio can be improved. Here, when there is an input from the row selection terminal 16, the potential of the photodiode 14 is sent to the memory array 4 through the output terminal 18 and the pixel data transfer line 7 of FIG.

In FIG. 2, n transistors are all used as MOS transistors.
Although MOS is used, some or all of them use pM
The same operation is performed by using an OS.

Next, how the unit pixel circuits 1 shown in FIG. 2 are connected in the pixel array 2 shown in FIG. 1 will be described. The row selection terminals 16 are connected by pixels having the same y coordinate arranged in the horizontal direction, and are collectively selected by the pixel array vertical scanning circuit 5 on a row basis. Output terminals 18 from the pixels are connected by pixels arranged at the same x-coordinate in the vertical direction, and connected to the pixel data transfer line 7.
Therefore, the pixel data transmitted on the pixel data transfer line 7 is
The row is selected by the pixel array vertical scanning circuit 5.

FIG. 3 shows an example of the structure of the unit memory cell 3. The pixel data output from the pixel array 2 through the pixel data transfer line 7 in FIG.
2, the capacitor 25 is charged through the memory cell selecting MOS transistor 24 when there is an input from the memory cell selecting terminal 23. With this operation, the pixel data is transferred to the memory cell. The potential of the capacitor 25 is input to the gate terminal of the amplifying MOS transistor 26, is current-amplified, and is switched to m (here, four) output destinations. Output destination selection MOS transistor 2
7, 28, 29 and 30 are output destination selection terminals 31, 32 and 3
The memory data, which is controlled by the control signals 3 and 34 and is current-amplified, is switched to the output terminals 35, 36, 37 and 38 for output.

In FIG. 3, pMOS is applied to the MOS transistor 26.
S, other MOS transistors 24, 27, 28, 29,
Although the nMOS 30 is used, a similar function can be obtained by replacing a part or all of the nMOS with an nMOS or a pMOS.

Next, how the unit memory cells 3 of FIG. 3 are connected in the memory array 4 shown in FIG. 1 will be described. The memory cell input terminals 22 are connected by memory cells having the same x coordinate arranged in the vertical direction, and are connected to the pixel data transfer line 7. The memory cell selection terminals 23 are connected between memory cells in the same row arranged in the horizontal direction, and are collectively selected by the memory array vertical scanning circuit 6 on a row basis. This makes it possible to transfer pixel data to the memory in units of rows by selecting a row in which data is to be written by the memory array vertical scanning circuit 6. The output terminals 35, 36, 37, and 38 are connected by the memory cells of the same row arranged in the horizontal direction, and it is possible to output m (in this case, 4) data from each row in parallel, and as a whole It is configured to obtain nxm output lines. Output destination selection terminals 31, 3
2, 33, and 34 are connected by memory cells of the same x coordinate arranged in the vertical direction, and x of the memory for outputting data is connected.
Coordinates (sets) can be selected collectively for each row. Thus, the data from the selected memory cell in the first column is output from the top of the output line of each row, and the data in the second column is output from the second output line. Are output in parallel in accordance with the position of

FIG. 4 shows an example of the structure of the kernel size x direction setting logic. 39 is a memory array output destination selection terminal, 40 is an AND gate, 41 is an output enable terminal, and 42 is an x coordinate input terminal for receiving an output from the x direction multiplexer 10 of FIG.

The operation of this circuit is as follows. First,
When an arbitrary x coordinate is selected by the x-direction multiplexer 10 in FIG. 1, one of the input terminals 42 becomes H.
At this time, if the output enable terminal 41 is set to H,
One output of the D gate 40 becomes H, and four of the memory array output destination selection terminals 39 connected thereto become H. This output selects the output destination selection terminals 31, 32, 33, and 34 for four consecutive columns from the designated x coordinate of the memory array 4. By this operation, it is possible to output 4 × 4 pieces of arbitrarily continuous four columns of pixel data in the memory cell to the parallel output 11 of the memory array 4.

Also, from the four terminals of the parallel output 11 corresponding to the output terminals 35 of the memory cells in each row, 4 ×
Each line of the parallel output 11 is associated one-to-one with the relative position of the memory cell within the 4 × 4 range, such that the data of the four memory cells having the smallest x coordinate in the range of 4 is output. be able to. And this correspondence is constant during operation.

The parallel output 11 is weighted and summed,
As the weighting block 12 for obtaining the filtering result, for example, one having a configuration as shown in FIG. 5 may be used.
The weight setting unit 43 is a current weighting circuit group that can give an arbitrary weight to the input current. These weighted outputs are added by the weighted output adder 44 to obtain the filtering output 13.

Next, the operation of the entire weighting block 12 of FIG. 5 will be described. First, as the initial setting, the kernel 4 ×
For each point in the range of 4, a corresponding weight setting unit 4
The weight of 3 is set according to the value of the kernel. Thereafter, the pixel data is transferred to the memory cells, and the horizontal scanning of the memory is sequentially performed. As a result, the result of filtering the 4 × 4 range of the pixel data is sequentially obtained from the output terminal 13.

Hereinafter, the overall operation of the image processing apparatus shown in FIG. 1 will be described. First, the pixels in each row are reset by the pixel reset shift register 8, and after a fixed image accumulation time, the image is transferred to the memory. When an arbitrary row is selected by the pixel array vertical scanning circuit 5, the pixel data of the row, that is, the potential of the photodiode in the unit pixel circuit 1 is transmitted through the pixel data transfer line 7. At this time, when an arbitrary row of the memory array 4 is selected by the memory array vertical scanning circuit 6, the capacitor 25 of each memory cell 3 is charged to a potential according to the pixel data. Thereby, any pixel data 1 in the pixel array 2
The row is transferred to any one row in the memory array 4. By repeating this operation four times, any four rows in the pixel array 2, for example, four rows continuous from a certain y coordinate are transferred to the memory array 4 in row units.

Next, horizontal scanning is performed. By sequentially shifting the x-direction multiplexer 10 and setting the output enable terminal 41 of the kernel size x-direction setting logic 9 to H at each time, 4 × 4 pixels from a set of four columns of continuous memory cells are set. Data can be taken out at the same time as the parallel output 11 of the memory cell. The range to be extracted is 4 × 4 because the x-direction multiplexer 10 is shifted one by one in the horizontal direction by sequentially shifting.
Can be easily scanned in order on the memory.

The 4 × 4 pixel data extracted as the parallel output 11 is subjected to weighting and sum calculation by the weighting block 12, and as a result, the result obtained by filtering the 4 × 4 range of the image is output. It is obtained sequentially from terminal 13.

As described above, in the present embodiment, the pixel data of the n × m unit pixel circuits in the pixel array can be simultaneously taken out, a given parameter can be given, and the filtering operation can be performed. The two-dimensional filtering while scanning the pixel array by sequentially shifting the range of xm can be performed with a small number of procedures. In addition, since the output is handled by current, addition of data at the time of filtering can be realized by a simple mechanism that only connects lines. Further, since a memory unit is provided and the pixel array and the operation unit are divided into regions, a simple pixel structure is obtained, and the size of the pixel can be reduced.

Although the pixel array 2 and the memory array 4 are arranged side by side on the same plane as shown in FIG. 1 in the present embodiment, a configuration in which the pixel array 2 and the memory array 4 are stacked may be adopted. Can be reduced in size.

Embodiment 2 In the first embodiment, as shown in FIG. 4, which can select four pixels continuous in the horizontal direction, is used as the kernel size x direction setting logic 9, but if it is desired to change the selection method in the horizontal direction, as shown in FIG. Other forms such as this can also be used.

The operation of this circuit is as follows. First,
When an arbitrary x coordinate is selected by the x-direction multiplexer 10 in FIG. 1 and the output enable terminal 41 is set to H, one output of the AND gate 40 becomes H, and the output of the memory array output destination selection terminal 39 connected to it becomes H. Are H. This output selects the respective output destination selection terminals 31, 32, 33, and 34 for every other four columns from the designated x coordinate of the memory array 4. With this operation, 4 × 4 pieces of pixel data of four columns taken every other position from an arbitrary position in the memory cell can be output to the parallel output 11 of the memory array 4.

As described above, in the present embodiment, it is possible to give a shape different from that of the first embodiment with respect to the horizontal kernel size.

Embodiment 3 Embodiments 1 and 2 above
In the above, there is shown a kernel size x direction setting logic 9 that can select four columns that are continuous in the horizontal direction and a column that can select every other four columns. Can be used as shown in FIG.

In the circuit shown in FIG. 7, of the two consecutive four-column selection enable terminals 45 and the alternate four-column selection enable terminals 46, only the continuous four-column selection enable terminal 45 is set to H.
The same operation as the circuit of FIG. 4 is performed, and the same operation as that of the circuit of FIG.

As described above, in the present embodiment, a plurality of methods for the horizontal kernel size can be electrically switched and used by the same circuit. Also, the pattern in the horizontal direction of the kernel can be set to any access pattern based on the same concept as well as four consecutive rows and every other four rows.

Embodiment 4 In the first embodiment, a case has been described in which the output from the memory array is arbitrarily weighted. However, there are many points that have the same weight in the kernel, and these are collectively subjected to two-dimensional filtering with a smaller weighting circuit. If it is desired to realize this, a configuration as shown in FIG. 8 can be used.

FIG. 8 is a diagram showing a configuration used for the weighting block 12. As shown in FIG. N × m from memory array 4
This is a part for performing a filtering operation by weighting the output of the book. It is assumed that there are k kinds of weights (excluding 0) in a kernel to be calculated (k: natural number, here, two kinds). Parallel output 11 from memory array 4 of FIG.
Are passed through the same number of weight selection units 47,
Switching between weight a, weight b, or no weight (weight 0). The switched output is the weight selection output coupling unit 4
The sum is added by combining at 8,49. The added currents are weighted by weights a and b by weight setting sections 50 and 51, respectively, and the final filtering output is obtained from output terminal 52 by adding the outputs.

FIG. 9 shows an example of the structure of the weight selection unit 47. In the figure, a weight selection unit input terminal 53
Is one of the outputs of the memory array 4. The outputs of the weight selection memories 54 and 55 are input to the gate terminals of the weight selection switch transistors 56 and 57, and switch the output destination of the current of the input terminal to the weight selection output terminals 58 and 59. The switched output is connected to the output terminals 58 and 59 and the coupling unit 4 in the weighting block 12 shown in FIG.
The weights are linked to weights a and b via 8 and 49, respectively.

Next, the weight selection unit 4 shown in FIG.
7 will be described. First, the weight selection memory 5
4, 55 are set to one H, one L, or both L. When the weight selection memory 54 is set to H, the current passing through the input terminal 53 is connected to the weight setting unit 50 through the weight selection switch transistor 56. When the weight selection memory 55 is set to H, the current flowing through the input terminal 53 is connected to the weight setting unit 51 through the weight selection switch transistor 57. When both are set to L, the current supplied to the input terminal 53 is separated from the output, which is equivalent to giving a weight of 0.

Next, the operation of the entire weighting block 12 of FIG. 8 will be described. First, as the initial setting, the kernel nx
For the point in the range of m where the weight a is to be given, the weight selection memories 54 and 55 of the corresponding weight selection unit 47
Are set to H and L, respectively. Similarly, L and H are assigned to the point at which the weight b is to be given, and L and L are assigned to the point at which the weight is to be set to 0. Thereafter, as in the first embodiment, the transfer of the pixel data to the memory array and the horizontal scanning of the memory array are sequentially performed, and the result of filtering the image with a kernel of size n × m is sequentially obtained from the output terminal 52. Can be

As described above, in the present embodiment,
The number of weight setting units can be reduced to a necessary minimum number while maintaining the property that weights can be given independently for each point.

Embodiment 5 FIG. In the fourth embodiment, when the y-coordinate range to be filtered is changed, it is necessary to newly transfer pixel data to the memory array 4. At this time, not all of the n rows are transferred, and new data is required. By transferring only the rows, the pixel data transfer time can be reduced. This is achieved by using n × m which is used to collect points having the same weight in the fourth embodiment.
A mechanism having a shift register configuration as shown in FIG. 10 is used as a mechanism for controlling the × k weight selection memories.

FIG. 10 shows a part of the circuit configuration used in the weighting block 12, and n parallel outputs 11 from the memory array 4 corresponding to the same x-coordinate (here, four).
The weight selection unit 47 of FIG. In the present embodiment, among the k × n (here, 2 × n = 8) weight selection memories of the n weight selection units 47, n units belonging to the same weight are connected cyclically. , Shift registers 60 and 61.

Next, the operation of FIG. 10 will be described with reference to FIG. FIG. 11 shows two-dimensional filtering performed on four consecutive rows. Of the data of the pixel array 2, four rows of y = 0, 1, 2, 3 have been transferred to the memory array 4, and the weighting block 62 has been set.
It is assumed that one horizontal scanning period of the two-dimensional filtering has already been completed, and the filtering output 63 has been extracted from the output terminal. When performing vertical scanning and transfer to the memory array 4 in order to filter the next row, all four rows of y = 1, 2, 3, and 4 of the data of the pixel array 2 are not transferred, and y = 4 is transferred to the row of the memory array containing the data of y = 0. Then, the shift registers 60 and 61 are all shifted by one bit, and the set value of the weight selection memory is shifted one by one in the y direction. When the horizontal scanning of the memory array 4 is performed in this state, the filtering output 65 of the weighting block 64 becomes y = 1,
This is equivalent to the result of filtering the four rows 2, 3, and 4 using the kernel given first. This is because both the memory cell 3 and the weight selection memory in the weighting block are shifted in the y direction by the same number of times.

The operation for moving to the next horizontal scanning period can be performed in exactly the same manner. Y of the data of the pixel array 2
= 5 is transferred to the row of the memory array 4 containing the data of y = 1, and all the shift registers in the weighting block are shifted by one bit. Thus, also in the present embodiment, vertical scanning can be realized by repeating the same operation.

As described above, in this embodiment, the weight selection memory in the weighting block can be shifted in the y direction, and the weight selection memory is shifted in synchronization with the pixel array vertical scanning circuit 5 and the memory vertical scanning circuit 6. The transfer amount of the pixel data to the memory cells during the dimensional filtering can be reduced to one row for one horizontal scanning period.

Embodiment 6 FIG. In the first embodiment, in order to perform two-dimensional filtering of an image, if pixel data in a range of n × m is to be sequentially output, for example, when n = 4, y is first stored in the memory array 4. = 0 to 3 and then y = 1 to 4 and then y = 2 to 5..., The pixel array vertical scanning circuit 5 2,3,1,
Scanning such as 2,3,4,2,3,4,5, ... must be performed. As a circuit configuration effective for such a purpose, the configuration shown in FIG. 12 can be used.

In FIG. 12, reference numerals 66 and 67 denote pixel array vertical scanning circuit shift registers, respectively.
Has the same number of bits as the number of pixels in the vertical direction. Here 6
6 is called a parent shift register and 67 is called a child shift register. 68 is a reset terminal of the parent shift register, 69 is a first-stage input terminal of the parent shift register, and 70 is a clock terminal of the parent shift register. By inputting a clock from the clock terminal 70, the contents of the parent shift register 66 are shifted down by one bit, and the value of the first stage input terminal 69 is read into the first stage. 71 is a data transfer line enable terminal for the child, 72 is a reset terminal of the child shift register,
73 is a clock terminal of the child shift register. By setting the transfer enable terminal 71 to H and inputting a clock from the clock terminal 73, the contents of the parent register 66 are copied to the child register 67. 74 is a pixel array access enable terminal by a child shift register;
Is a row selection terminal of the pixel array 2. By setting the pixel array access enable terminal 74 to H, the row corresponding to the H bit in the child register 67 is selected.

Next, the operation of the entire pixel array vertical scanning circuit 5 shown in FIG. 12 will be described. First, a value of 100 0... Is written to the parent register 66 by using the reset terminal 68, the first-stage input terminal 69, and the clock terminal 70. This is copied to the child register 67 using the transfer enable terminal 71 and the clock terminal 73. Here, when the pixel array access enable terminal 74 is set to H, the output of the row selection terminal 75 of the pixel array 2 becomes 100 0 0
, And one row of y = 0 is selected. Here, when the child register 66 is shifted, its contents become 0 1 0 0.
.., So that the pixel array access enable terminal 74 is appropriately set to H, so that y
= 0, 1, 2,... Furthermore, (1)
By following the procedure of clearing the child register, (2) shifting the parent register, and (3) copying again from the parent register to the child register, each row is selected as y = 1, 2, 3,.
Then select each row with y = 2,3,4, ...
Selections can be made according to a sequence that is piecewise continuous but returns halfway.

Further, another operation of the pixel array vertical scanning circuit 5 shown in FIG. 12 will be described. Child register 6
7 into 1 0 0 0 ..., 0 1 0 0 ..., 0 0
.., The timing of setting the pixel array access enable terminal 74 to H is set to one for every two shifts, for example, y = 0, 2, 4, 6,
Scanning such as 1, 3, 5, 7, 2, 4, 6, 8,... Can be performed. Using this operation for the vertical scanning in the first embodiment is equivalent to using a filter roughly 2n wide in the vertical direction as a kernel for two-dimensional filtering.

Further, when shifting the parent register 66, a procedure of shifting twice and copying to the child register 67 is performed, for example, y = 0, 2, 4, 6, 2,
Scanning such as 4, 6, 8, 4, 6, 8, 10, ... can be performed. Using this operation for the vertical scanning in the first embodiment is equivalent to accessing half the vertical resolution using only every other row of the pixel area at the time of two-dimensional filtering.

Further, based on the same concept, it is possible to set another arbitrary access pattern in the above-described four consecutive columns and every other four columns.

As described above, in the present embodiment,
Different vertical scanning patterns such as sequential access with a width of n pixels in the y direction, sequential access with a width of 2n pixels, and half-resolution access can be realized with a common structure and by repeating the same procedure.

Embodiment 7 FIG. Hereinafter, another embodiment of the present invention will be described. When it is desired to make the horizontal pitch of pixels finer for higher resolution, the configuration shown in FIG. 13 allows the horizontal pitch to be smaller than the physical size of the memory array.

FIG. 13 is a block diagram showing a layer processing apparatus according to the seventh embodiment of the present invention, which shows a light receiving element circuit array (pixel array), a memory array, and a drive circuit therefor. As in the first embodiment, a case where the number of pixels n × m to be accessed simultaneously is 4 × 4 is shown, and an 8 × 8 pixel array is shown. In the figure, a unit pixel circuit 1, a pixel array 2, a pixel array vertical scanning circuit 5, and a reset shift register 8 are the same as in the first embodiment. 3
Are the unit memory cells, 76 and 77 are the memory array (top) and the memory array (bottom) in which the unit memory cells are arranged, 7
8, 79 are memory array vertical scanning circuits (upper) and (lower) for selecting rows of the memory arrays 76, 77;
Reference numerals 0 and 81 denote pixel data transfer lines (upper) and (lower) for transferring pixel data to the memory arrays 76 and 77, respectively.
82 and 83 are kernel size x direction setting logic (top)
And (bottom), 84, 85 are x-direction multiplexers (top) and (bottom), 12 is a weighting block, and 13 is a filtering output. In the present embodiment, even columns and odd columns of the memory array 4 in the first embodiment are physically separated from each other. Therefore, the output destination of the pixel data transfer line is divided into an even column and an odd column. Since the total number of memory cells 3 is the same, the same operation as that of the first embodiment can be performed, but the configuration of the horizontal scanning circuit differs.

FIG. 14 shows an example of the structure of the kernel size x direction setting logic (upper) 82. 86 is a memory array output destination selection terminal, 87 is an output enable terminal (upper),
An x coordinate input terminal 88 receives an output from the x direction multiplexer (upper) 84 in FIG.

FIG. 15 shows an example of the structure of the kernel size x direction setting logic (lower) 83. 89 is a memory array output destination selection terminal, 90 is an output enable terminal (lower),
An x-coordinate input terminal 91 receives an output from the x-direction multiplexer (lower) 85 in FIG.

Next, the overall operation of the image processing apparatus shown in FIG. 13 will be described. The same operation is performed by the same control signal as in the first embodiment. Memory array vertical scanning circuits 78, 7
9. Kernel size x direction setting logic 82, 83, x
For the direction multiplexers 84 and 85, the same signal is given at the top and bottom. Thereby, the upper and lower memory arrays 76,
Of the parallel output from each row of 77, corresponding to each 4 × 4 point, data of the memory cell is output from one of the upper and lower two sets of lines, and one of them becomes high impedance.
Therefore, by connecting the upper and lower memory array outputs, data in a 4 × 4 range is output in parallel. By inputting this to the weighting block 12, a filtering output 13 is obtained as in the first embodiment.

As described above, in the present embodiment, by dispersing the memory array up and down, the pixel pitch can be reduced without changing the characteristics of the control signal and the output to be provided. The size can be reduced to half, which is effective for increasing the resolution of the image processing apparatus.

In this embodiment, a configuration in which one set of memory arrays and one output circuit are arranged vertically has been described.
The pixel pitch can be further reduced and the pixel array can be made finer, for example, by setting two pairs at the top and bottom.

Embodiment 8 FIG. In the first embodiment, when a part of the imaging surface is irradiated with strong light, there is a phenomenon that a virtual image is observed in the vicinity. This can be avoided by using a structure as shown in FIG. 16 for the unit pixel circuit 1.

FIG. 16 shows a structure of a unit pixel circuit according to the eighth embodiment of the present invention. In the drawing, reference numeral 92 denotes a power supply for amplifying pixel data, and 93 denotes a power supply for resetting a pixel. When strong light is applied to a part of the image, the potential of the photodiode in the unit pixel circuit in that part greatly decreases, and the current required for pixel reset increases. For this reason, if there is an interaction between the power supply line used for pixel reset and the power supply line used for pixel data amplification, the pixel reset operation becomes noise for the pixel read operation. FIG.
Separates both power supplies to avoid this.

As described above, in this embodiment, by separating the pixel reset power supply line and the pixel data amplification power supply line, there is no interaction between reset and readout, and a virtual image can be avoided.

[0073]

As described above, according to the first configuration of the present invention, a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling the output of the element are arranged in a two-dimensional array. A memory cell having a two-dimensional pixel array, a pixel data memory, and a control circuit for controlling input / output of the memory is arranged in the number of columns of the pixel array in the horizontal direction, and in n rows (n: natural number) in the vertical direction An array of pixel data memory arrays, a pixel array vertical scanning circuit for selecting a row in the two-dimensional pixel array, and a memory array vertical scanning circuit for selecting a row in the pixel data memory array in synchronization with the pixel array vertical scanning circuit A row selected by the memory array vertical scanning circuit from a plurality of unit pixel circuits in a row selected by the pixel array vertical scanning circuit in row units; Transfer to a plurality of memory cells,
A data transfer unit that stores the pixel data in the pixel data memory; and a horizontal scanning circuit that selects m columns (m: natural number) of memory cells that output from the pixel data memory array. Since the image processing apparatus is constituted by the output means for taking out the respective outputs from the m memory cells in parallel, the two-dimensional filtering while taking out the image is realized by a simple pixel structure by calculating the parallel outputs. it can.

Further, according to the second configuration of the present invention, each memory cell selects m (m: natural number, m ≧ 2) output terminals and the output terminal serving as an output destination of each memory cell. The output means connects the m output terminals to each other between the memory cells in the same row arranged in the horizontal direction to form n × m output lines, and The m selection terminals are connected to each other between the memory cells of the same column arranged in a row, and a parallel output corresponding to the position of the selected m columns of memory cells is taken out from the n × m output lines. It is possible to output nxm data in parallel from nxm memory cells in the memory array.

Further, according to the third configuration of the present invention, the horizontal scanning circuit is configured to control the number X (X:
A natural number), and a logic circuit for selecting m columns of the pixel data memory array according to a variable access pattern with the position where the input was made as a reference point. The size and shape of the directions can be freely changed.

According to the fourth configuration of the present invention, each of the n × m outputs from the output means is weighted,
Since an output circuit that takes the sum of these weighted outputs is provided, a weighted average can be output from n × m memory cells in the memory array.

Further, according to the fifth configuration of the present invention, the output circuit collectively outputs the same weight.
Since means for adding these outputs before weighting is provided, the number of weighting circuits can be reduced.

Further, according to the sixth configuration of the present invention, the output circuit is configured such that, for one output from the output means, k switches (k: natural number) and one bit connected to each switch are provided. A weight selection unit comprising a storage device of
Since the output from the output means is distributed to k weight setting units by this weight selection unit, any of the k weights can be independently and arbitrarily given to each output of the memory array.

Further, according to the seventh configuration of the present invention, the storage device connected to each of the switches has a structure that can be cyclically shifted in the vertical direction in the corresponding memory cell. The number of times of transfer of the pixel data to the memory cell can be reduced, and the driving can be simplified.

Further, according to the eighth configuration of the present invention, the pixel array vertical scanning circuit selects a row of the pixel array according to a variable access pattern, using an arbitrary row in the vertical direction of the pixel array as a reference point. Since the logic circuit is provided, the size and shape of the kernel in the vertical direction can be freely changed.

Further, according to the ninth configuration of the present invention, the pixel array vertical scanning means has a structure in which two shift registers are overlapped, each shift can be shifted independently, and one of the shifts can be shifted from the other. Since the structure is such that values can be successively copied, vertical scanning for transferring pixel data to memory cells by n rows at the time of two-dimensional filtering becomes easy and fast.

According to the tenth structure of the present invention,
Since the pixel data memory array is divided into two parts, a memory group corresponding to the even columns and a memory group corresponding to the odd columns of the two-dimensional pixel array, and a horizontal scanning circuit is provided for each memory group. A pixel array having a pitch smaller than the minimum pitch of the memory array or the horizontal scanning circuit can be realized.

According to the eleventh structure of the present invention,
In the unit pixel circuit, wiring is divided between a power supply for charging a photoelectric conversion element and a power supply for a control circuit for controlling the output of the photoelectric conversion element. Even if a large pixel reset current flows,
There is no noise in reading neighboring pixel data.

According to the image processing method of the present invention,
A pixel data memory and an input / output of the memory are controlled for a two-dimensional pixel array in which a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling the output of the element are arranged in a two-dimensional array. A pixel data memory array in which a plurality of memory cells having a control circuit are arranged in a two-dimensional array is provided, and a plurality of unit pixel circuits arranged in a row direction in the pixel array are selected to perform pixel data row by row. Transfer to each memory cell in a predetermined row of the pixel data memory array, store the pixel data in the pixel data memory, and select a plurality of columns of memory cells that output from the pixel data memory array. Since each output from the memory cell is taken out in parallel, two-dimensional filtering while taking out an image is performed.
It can be realized with a simple pixel structure.

[Brief description of the drawings]

FIG. 1 is a diagram showing a light receiving element circuit array, a memory array, and peripheral circuits constituting an image processing apparatus according to a first embodiment of the present invention;

FIG. 2 is a diagram showing a structure of a unit pixel circuit used in the light receiving element circuit array according to the first embodiment of the present invention.

FIG. 3 is a diagram showing a structure of a unit memory cell used in the memory array according to the first embodiment of the present invention;

FIG. 4 is a diagram showing a structure of a kernel size x direction setting logic according to the first embodiment of the present invention;

FIG. 5 is a diagram showing a structure of a weighting block according to the first embodiment of the present invention.

FIG. 6 is a diagram illustrating a structure of a kernel size x direction setting logic according to the second embodiment of the present invention;

FIG. 7 is a diagram showing a structure of a kernel size x direction setting logic according to a third embodiment of the present invention.

FIG. 8 is a diagram showing a structure of a weighting block according to Embodiment 4 of the present invention.

FIG. 9 is a diagram showing a structure of a weight selection unit used for a weighting block according to Embodiment 4 of the present invention.

FIG. 10 is a diagram showing a structure of a weight selection unit used for a weighting block according to Embodiment 5 of the present invention.

FIG. 11 is a diagram showing an operation of the image processing device according to the fifth embodiment of the present invention.

FIG. 12 is a diagram showing a structure of a pixel array vertical scanning circuit according to a sixth embodiment of the present invention.

FIG. 13 is a diagram showing a light receiving element circuit array, a memory array, and peripheral circuits constituting an image processing apparatus according to a seventh embodiment of the present invention.

FIG. 14 is a diagram showing a kernel size x direction setting logic (upper) according to the seventh embodiment of the present invention.

FIG. 15 is a diagram showing logic (bottom) for setting a kernel size in the x direction according to the seventh embodiment of the present invention;

FIG. 16 is a diagram showing a structure of a unit pixel circuit according to an eighth embodiment of the present invention.

FIG. 17 is a diagram illustrating a light-receiving element circuit array and a peripheral circuit constituting a conventional image processing apparatus.

FIG. 18 is a diagram showing a unit pixel circuit of a conventional light receiving element circuit array.

FIG. 19 is a diagram showing a light receiving element circuit array constituting another conventional image processing apparatus.

[Explanation of symbols]

1 unit pixel circuit, 2 pixel array, 3 unit memory cell, 4 memory array, 5 pixel array vertical scanning circuit,
6 memory array vertical scanning circuit, 7 pixel data transfer line, 8 pixel reset shift register, 9 kernel size x direction setting logic, 10 x direction multiplexer, 11 parallel output, 12 weighting block, 13
Filtering output, 14 pixel data storage photodiode, 15 amplifying MOS transistor, 16
Row selection terminal, 17 Row selection MOS transistor, 18
Output terminal, 19 resetting MOS transistor, 2
0 reset terminal, 21 pixel unit power supply, 22 memory cell input terminal, 23 memory cell selection terminal, 24 memory cell selection MOS transistor, 25 capacitor, 2
6 MOS transistors for amplification, 27, 28, 29, 30
MOS transistors for output destination selection, 31, 32, 3
3, 34 output destination selection terminal, 35, 36, 37, 38
Output terminal, 39 memory array output destination selection terminal, 40
AND gate, 41 output enable terminal, 42 x-coordinate input terminal, 43 weight setting unit, 44 weighted output adder, 45 continuous four column selection enable terminal, 46
Every other four column selection enable terminal, 47 weight selection unit, 48, 49 weight selection output coupling section, 50, 51
Weight setting unit, 52 filtering output terminal, 53 weight selection unit input terminal, 54, 55 weight selection memory, 56, 57 weight selection switch transistor, 5
8, 59 weight selection output terminal, 60, 61 shift register, 62 weight block (before shift), 63 filtering output (before shift), 64 weight block (after shift), 65 filtering output (after shift), 66 pixels Array vertical scanning circuit parent shift register, 67 pixel array vertical scanning circuit child shift register,
68 parent shift register reset terminal, 69 parent shift register first stage input terminal, 70 parent shift register clock terminal, 71 data transfer line enable terminal, 72
Child shift register reset terminal, 73 child shift register clock terminal, 74 pixel array access enable terminal, 75 pixel array row selection terminal, 76 memory array (top), 77 memory array (bottom), 78 memory array vertical scanning circuit (top) , 79 memory array vertical scanning circuit (bottom), 80 pixel data transfer line (top), 8
1 pixel data transfer line (bottom), 82 kernel size x direction setting logic (top), 83 kernel size x
Direction setting logic (bottom), 84 x direction multiplexer (top), 85 x direction multiplexer (bottom), 86 memory array output destination selection terminal (top), 87 output enable terminal (top), 88 x coordinate input terminal (top), 89 memory array output destination selection terminal (lower), 90 output enable terminal (lower), 91 x coordinate input terminal (lower), 92 pixel data amplification power supply, 93 pixel reset power supply, 94 unit pixel circuit, 95 vertical scanning circuit , 96 horizontal scanning circuit, 97 difference circuit, 98 pixel data storage photodiode, 99 reading transistor, 100 column selection terminal, 101 row selection terminal, 102 output terminal, 1
03 unit pixel circuit, 104 pixel array, 105 analog memory array, 106, 107, 108 clock signal, 109, 110, 111 memory array output.

Claims (12)

[Claims] 1. A two-dimensional pixel array in which a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling the output of the element are arranged in a two-dimensional array, a pixel data memory, and an input / output of the memory. A memory cell having a control circuit for controlling a pixel data memory array in which the number of columns of the pixel array is arranged in the horizontal direction and n rows (n: natural number) are arranged in the vertical direction, and a row in the two-dimensional pixel array is selected. And a memory array vertical scanning circuit for selecting a row in the pixel data memory array in synchronization with the pixel array vertical scanning circuit, and a plurality of rows selected by the pixel array vertical scanning circuit. The pixel data is transferred from the unit pixel circuits to a plurality of memory cells in a row selected by the memory array vertical scanning circuit in row units, and the pixel data is A data transfer means for storing the data in the pixel data memory; and a horizontal scanning circuit for selecting m columns (m: natural number) of memory cells for outputting from the pixel data memory array. An image processing apparatus comprising output means for taking out respective outputs from memory cells in parallel. 2. Each memory cell has m (m: natural number, m ≧ 2)
2) It has m output terminals and m selection terminals for selecting the output terminals to be output destinations of each memory cell, and the output means comprises:
The m output terminals are connected to each other between memory cells in the same row arranged in the horizontal direction to form n × m output lines, and the m output terminals are selected between memory cells in the same column arranged in the vertical direction. 2. The image processing apparatus according to claim 1, wherein the terminals are connected to each other, and a parallel output corresponding to a position of the selected m columns of memory cells is taken out from the n × m output lines. 3. The horizontal scanning circuit has X input terminals corresponding to the number X (X: natural number) of memory cells in a horizontal direction, and uses a position where an input is made as a reference point to form a variable access pattern. 3. The image processing apparatus according to claim 1, further comprising a logic circuit for selecting m columns of the pixel data memory array. 4. An output circuit according to claim 1, further comprising an output circuit for weighting each of the n × m outputs from the output means, and obtaining a sum of the weighted outputs. An image processing apparatus according to claim 1. 5. The image processing apparatus according to claim 4, wherein the output circuit includes means for collecting outputs to which the same weight is given and adding these outputs before weighting. 6. The output circuit has, for one output from the output means, k (k: natural number) switches and a weight selection unit including a 1-bit storage device connected to each switch. 6. The image processing apparatus according to claim 5, wherein the output from the output unit is distributed to k weight setting units by the weight selection unit. 7. The image processing apparatus according to claim 6, wherein the storage device connected to each switch has a structure capable of cyclically shifting in a vertical direction in a corresponding memory cell. 8. The pixel array vertical scanning circuit includes a logic circuit that selects a row of the pixel array according to a variable access pattern, using an arbitrary row in the vertical direction of the pixel array as a reference point. Item 8. The image processing device according to any one of Items 1 to 7. 9. The pixel array vertical scanning circuit has a structure in which two shift registers are stacked, and each shift register can be shifted independently.
9. The image processing apparatus according to claim 8, wherein a value can be copied from one run to one run. 10. A pixel data memory array is divided into two parts, a memory group corresponding to an even column and a memory group corresponding to an odd column of a two-dimensional pixel array, and a horizontal scanning circuit is provided for each memory group. The image processing apparatus according to claim 1, wherein the image processing apparatus is provided. 11. The unit pixel circuit according to claim 1, wherein a power supply for charging the photoelectric conversion element and a power supply for a control circuit for controlling the output of the photoelectric conversion element are separated from each other. An image processing device according to any one of the above. 12. A two-dimensional pixel array in which a plurality of unit pixel circuits each having a photoelectric conversion element and a control circuit for controlling the output of the element are arranged in a two-dimensional array. A pixel data memory array in which a plurality of memory cells each having a control circuit for controlling input / output are arranged in a two-dimensional array is provided, and a plurality of unit pixel circuits arranged in a row direction in the pixel array are selected to form a pixel. Data is transferred to each memory cell in a predetermined row of the pixel data memory array on a row basis, and the pixel data is stored in the pixel data memory, and a plurality of columns of memory cells for outputting from the pixel data memory array are provided. An image processing method, wherein each output is selected and the respective outputs from the selected memory cells are taken out in parallel.