JP2008526062A - Image sensor with totally separate color zones - Google Patents

Abstract

  The present invention relates to an electronic color image sensor, and more particularly to a very small sensor for manufacturing a small photographing device or camera (which can be incorporated into a mobile phone or the like). This sensor has an optical system for projecting an image of a scene to be observed onto a series of photosensitive areas, which are manufactured on the same monolithic semiconductor chip integrated with the optical system. Yes. The optical system comprises a plurality of optical subassemblies (L1, L2, L3, L4), and the series of photosensitive areas is divided into at least two matrices (M1, M2, M3, M4) that can be individually read, The sub-assemblies are designed to project the entire scene to be observed onto each matrix, and above the various matrices are two uniform filters (F1, F2 of different colors) for passing a single color of light through each matrix. , F3, F4) are arranged respectively.

Description

The present invention relates to an electronic color image sensor, and more particularly to a very small sensor for manufacturing a small photographing device or camera (which can be incorporated into a mobile phone or the like).

It would be desirable to produce the entire camera in a process that is as inexpensive as possible while meeting the increasing resolution, colorimetric quality, and compactness requirements.

  The color image sensor can be manufactured by the following method. That is, the surface of the silicon wafer is subjected to a number of steps such as masking, impurity implantation, temporary or final layer deposition of various compositions, etching of these layers, heat treatment, and the like. These steps are used to form a matrix of photosensitive spots and a circuit for processing the electrical signals associated with these photosensitive spots. Next, a color filter layer is deposited on the surface of the silicon wafer, and these are individually etched to form a matrix pattern. The matrix includes groups of three or preferably four color filters of different colors for each pixel of the sensor arranged in rows and columns. Above each photosensitive area, a basic filter that receives light of a single color is located. Adjacent filters located above adjacent photosensitive spots have different colors. Thus, each pixel is essentially provided with four color filters (generally two green and one red and blue) above four adjacent photosensitive areas that form the pixel.

  When the resolution is high, the proximity of adjacent areas is very large, and there is a risk that considerable optical crosstalk occurs. The reason for this is that a part of the light directed to a certain photosensitive area hits the adjacent area, or a part of the electrons photogenerated in the photosensitive area is actually captured by the adjacent area. It is done. As a result, of course, the spatial resolution is lost to some extent, which mainly affects image scenes with a high spatial frequency. However, in terms of color measurement, it can be seen that such an optical crosstalk phenomenon between adjacent regions is particularly serious. In other words, even image areas that have only a low spatial frequency (for example, a uniform red image area) may be affected (closer pixels corresponding to colors other than red were directed to those pixels. Since red electrons are systematically detected, the red color systematically deteriorates). The electronic image created by the uniform red area is no longer red but contains green and blue components.

  This optical crosstalk causes colorimetric degradation and is particularly likely to occur in CMOS technology. This is because in the case of CMOS technology, if the layer located between the color filter and the photosensitive region is high (several microns), crosstalk occurs.

  There are other color measurement problems in color image sensors, especially the moire problem. This problem is the result of the interpolation process performed to give the luminance value of the first color to a pixel that does not correspond to the first color and is located between the two pixels of the first color. appear.

  Patent FR-A-2 829 289 has already proposed a structure that reduces the problem of colorimetric quality degradation by having a thin substrate.

  It is an object of the present invention to provide another method for improving colorimetry. This method is applicable to both conventional structures and thin substrate structures, and is applicable regardless of the technology used (whether CMOS or not).

  The subject of the present invention is a color image sensor, the image sensor having an optical system for projecting an image of a scene to be observed onto a series of photosensitive areas, and the same monolithic semiconductor chip integrated with the optical system An image sensor having a series of photosensitive areas fabricated thereon, the series of photosensitive areas being divided into at least two matrices that are individually readable, the optical system comprising a plurality of optical subassemblies, The assembly is designed to project the entire scene to be observed onto each matrix, and two uniform filters of different colors are placed to pass a single color of each matrix light, one of which is above the first matrix The other one is characterized in that it is arranged above the second matrix.

  Thus, adjacent pixels of the sensor are not covered with different color filters, but each of the entire matrix is covered with a uniform color filter different from the adjacent matrix filter.

  Today, it is known to produce an optical system that has a very short focal length and can project a focused image of a scene onto a matrix of hundreds of thousands of pixels with dimensions of a few millimeters per side. is there. These optical systems are created and integrated into the sensor during the batch production of the sensor. That is, the sensors are collectively made on the silicon wafer, and then the wafer is divided into individual sensors. As a result, it is possible to collectively manufacture a plate including a large number of optical subassemblies that are positioned with high accuracy relative to each other, to collectively manufacture silicon wafers with various sensors, and to form a matrix of the optical subassemblies. As with the case, positioning with high accuracy is possible. The plate and wafer can be juxtaposed so that the optical subassembly and the respective photosensitive matrix face each other. The wafer / plate assembly is then divided into individual sensors. Each sensor has a plurality of optical subassemblies and a plurality of matrices so that a complete image of the observed scene can be received in a uniform color for each matrix.

  In practice, four matrices are used. The four matrices are arranged in a square shape when the matrix shape is square, and are arranged in a rectangular shape when the matrix shape is rectangular. The two matrices arranged along the square or rectangular diagonal are associated with the green filter, and the other two matrices arranged along the other diagonal are one of which is a red filter and the other is a blue filter. Associated.

  Signals output from various matrices of the same sensor chip are combined to represent the entire image in the various colors it has. The combining step is not in the luminance values received by a plurality of adjacent photosensitive regions in the same matrix, but in a plurality of (four locations) located in different matrices and in the same relative position in the various matrices. It only involves assigning the luminance values received by multiple (four) regions to the same point (a point containing multiple color components) in the electronic color image. The relative position considers imperfect positioning. That is, there may be a shift between the center of the optical subassembly and the corresponding center of the matrix, and this shift is believed to be different for the various optical subassemblies.

  According to the present invention, the problem relating to colorimetry generated in the prior art can be solved at least partially in a specific case. This is because the crosstalk generated between adjacent photosensitive regions is applied only between pixels of the same color. As in the prior art, this crosstalk is manifested by an inevitable reduction in resolution, but this is unavoidable and inevitably results in a colorimetric deterioration.

  In this solution, the image to be observed is theoretically projected onto the various matrices in the same way, that is, any image point of the scene to be observed is projected at the same relative position on the four matrices. It is assumed that However, it will be understood later that if the accuracy of the optical assembly is not sufficient to guarantee this identity of the relative position, electronic correction means can be found to at least partially compensate for these position errors.

  In particular, the matrix is preferably larger than the image of the scene to be observed. In such a method, positioning errors are calibrated if the center point in the observed scene is not exactly projected to the center of the four matrices due to the relative misalignment between the optical subassembly and the matrix. It is possible to electronically shift the signals output by the various matrices after the correction according to the observed error without losing image fragments. Even if the center of the image is not exactly located at the center of these matrices, the entire image of the scene to be observed remains projected on the four matrices.

  Only if the distance from the object to the sensor is much greater than the distance between the centers of the two optical subassemblies, the two optical subassemblies arranged in parallel to the same two matrices arranged in parallel Note that the projection of the image of the object is similarly centered on the two matrices. When the object is located at a non-infinity distance, the images projected on the two matrices tend to move away as the object is closer. The shift can be calculated as a function of the distance of the object (for a given distance of the optical subassembly), and taking this shift into account makes the images provided by the various matrices point-to-point enabled. be able to. The relative shift between images in two adjacent matrices is proportional to the spacing between the corresponding optical subassemblies and inversely proportional to the distance of the object.

  According to an important feature of the present invention, this shift can be calculated if the sensor comprises two matrices for green, one matrix for red and one matrix for blue. In this case, it is required to take an image of an object located relatively close (for example, a distance less than N times the focal length of the matrix, where N is the number of pixels in the line of the image projected onto the matrix). In some cases, the fact is used that the two optical subassemblies corresponding to the green matrix are separated by the same distance separating the subassemblies associated with the red and blue matrices. The goal is to determine how much the two green matrix images are optimally superimposed by the image correlation algorithm, so that to establish an image correspondence between the four matrix pixels, The shifts that must be applied to the various matrices of blue and red are estimated.

  According to another aspect of the invention, instead of using two green matrices, one red matrix, and one blue matrix to improve colorimetry, the four matrices have four different colors. You may set so that it may each be covered with a filter (especially red, green, blue, cyan filter).

  Other features and advantages of the present invention will become apparent upon reading the following detailed description with reference to the accompanying drawings.

FIG. 1 is a plan view showing a general arrangement example of pixels of a photosensitive matrix of a color image sensor based on CMOS technology. Although only a small number of pixels are shown, it should be understood that the matrix can be comprised of hundreds of thousands of pixels (eg, 1.2 million pixels). In this example, the photosensitive area of these pixels has an approximately octagonal shape. Pixels are generally controlled by horizontal and vertical conduction lines (not shown). These conduction lines are not perfect straight lines, but follow an octagonal outline.

FIG. 1 also shows the color organization. The letter R associated with the pixel is individually covered with a red (R) filter, the letter G is covered with a green (G) filter, and the letter B is covered with a blue (B) filter. Indicates that Adjacent pixels are arranged so that colors are alternated, and there are twice as many green pixels as red pixels and blue pixels.

FIG. 2 is a cross-sectional view schematically showing how a color image sensor (here, a thin film silicon sensor) is manufactured at a smaller scale than FIG. 1, and the color image sensor includes the following elements. ing.
A base 10 with an electrical connection 12;
A matrix of photosensitive areas formed in a very thin silicon substrate 20 (thickness is about 10 microns).
A color filter matrix layer 30 in which adjacent pixels are covered by filters of different colors R, G, B as shown in FIG.
A visual image forming system 40 that can project the entire image of the scene to be observed onto the photosensitive matrix (via a color filter); The optical system is composed of one or more transparent plates. These transparent plates are used to form one or more glass or transparent plastic lenses stacked on top of each other (here only a single lens L is shown). In order to focus the image of the observed scene on the surface of the matrix, it is necessary to position the optical system very accurately in the height direction above the photosensitive matrix. A transparent separating layer 35 formed to an appropriate thickness to ensure this precise positioning is shown in FIG.

FIG. 3 is a sectional view of a sensor according to the present invention, and FIG. 4 is a plan view of the sensor.

The matrix of photosensitive spots is divided into a plurality of matrices on the same integrated circuit chip, and the optical system is also divided into the same number of optical subassemblies. Each optical subassembly projects a complete view of the entire scene to be observed onto each matrix. Each matrix is covered by a single filter of uniform color.

In a preferred embodiment, four matrices are arranged in a square or rectangular format (two rows of two matrices each). These four matrices are represented by M1, M2, M3, M4 and are covered by filters F1, F2, F3, F4 and optical subassemblies L1, L2, L3, L4 (drawn as lenses), respectively. ing. The matrices M1 and M4 are opposed diagonally, and the matrices M2 and M3 are in the same positional relationship. Two of the filters F1 and F4 (here M1 and M4) in the diagonally opposite matrix are preferably green. The remaining two matrix filters are red (F2 on matrix M2) and blue (F3 on matrix M3), respectively. In FIG. 3, only filters F1 and F2, optical subassemblies L1 and L2, and matrices M1 and M2 are shown.

The optical subassemblies project substantially the same visible scene onto each matrix (as will be shown later, since the matrices are offset laterally from each other, the shift range is small).

Although the matrix is shown as a square matrix, the matrix can be rectangular if the photograph is taken in a rectangular format.

In high-precision product manufacturing with micromachine and microassembly technology, the relevant optical subassemblies are identical, have the same focal length, and are accurately and equally positioned when viewed with respect to the distance from the common plane of the four photosensitive matrices. Can be regarded as being.

In this case, assuming that the image of the scene to be observed is a white rectangular test pattern representing the theoretical outline of the scene, this outline is represented by four dotted rectangles shown in FIG. The same rectangular contour is projected onto each matrix. Here, a simple assumption is made that there is no distortion in the image (if there is distortion, the distortion is the same in all optical subassemblies).

Dotted rectangles are not necessarily positioned in the same way with respect to the photosensitive matrix onto which they are projected. The first reason for this is that the optical subassembly is similar to two people taking a photo in the same direction side by side looking at an image shifted in proportion to the camera shift. Due to the fact that they are not "seeing" the rectangle at exactly the same position because they are offset laterally from each other. Here, the shift of the optical subassembly is a few millimeters. The resulting rectangular shift is obtained by multiplying the sensor shift by the magnification. In this case, the magnification is defined as the ratio of the size of the image of the object projected onto the matrix to the size of the object itself. Thus, if the scene is spread at infinity, the shift is zero. In addition, due to product manufacturing accuracy limitations when transferring subassemblies onto the matrix, the optical subassemblies may not be positioned horizontally with respect to the matrix as precisely as needed. For this reason, the center of the subassembly is not always exactly perpendicular to the center of the matrix. On the one hand, there is a shift due to the object being a relatively close distance, on the other hand, there is an essential shift due to incomplete positioning of the optical subassembly relative to the matrix.

However, when the optical components are the same and the optical components are arranged at the same distance from the surface of the photosensitive area on which the rectangle is projected, the size of the projected rectangle is exactly the same for all rectangles.

Thus, at least the essential relative shift of the various matrices can be determined by in-factory calibration or subsequent calibration. A simple image of a small number of white spots arranged at a white reference test pattern (eg, the rectangle described above) or a sufficient distance (a distance greater than N times the focal length for a matrix composed of N pixel rows) Generates an image on each matrix, and once the corresponding relationship is established from the four images by the transformation vector, it corresponds point-to-point with any four images of any scene observed at a sufficient distance .

FIG. 5 is a plan view of the matrix M1, illustrating the calculation of the essential relative shift between the matrices.

The optical center of the optical subassembly that is vertically above the matrix M1 is indicated by O1. This subassembly projects a rectangular image IM1 of an object centered on the center axis O of the sensor at infinity onto a matrix. Since the center of the projected image is perpendicular to the point O1, it coincides with this point O1 in the plan view shown in FIG. If the alignment of the optical subassembly and the matrix is perfect, the center C1 of the matrix M1 itself can be considered to coincide with the point O1. For simplicity, the center C1 can be regarded as the intersection of the center row and the center column of the effective portion of the matrix M1. Due to imperfections, the center C1 may be slightly offset with respect to the center O1. The exact position of the rectangle IM1 and the exact position of the center O1 with respect to the center C1 of the matrix are referred to in a coordinate system consisting of pixel rows and pixel columns of the matrix. This reference process is performed by calibration in the factory. This reference operation is carried out for the other matrices using the images IM2, IM3, IM4, which are the same rectangle projected onto those matrices at infinity.

For the matrix M1, the difference in position along the x-axis and y-axis between the points O1 and C1 and for the other matrices, the points O2 and C2, the points O3 and C3, the points O4 and C4, and the x-axis Differences in position along the axis must be generated in order to match each image point in the observed scene with the corresponding point in each matrix when reading the electrical signal output by each matrix. Determine each shift along and each shift along the column. It is assumed that the scene is at infinity.

These shifts are stored in the calibration register. Thus, these registers contain information corresponding to the relative shift between the center of the optical subassembly and the centers of the various matrices. The electrical signals output by the four matrices are combined according to the contents of the registers in order to make the correspondence between the images output by the various matrices accurate and regular to compensate for the shift.

The electrical signals output by the various matrices are collected separately (although preferably synchronized).

The matrix M1 and the matrix M4 output signals representing green components in the observed scene image. The matrix M2 outputs the red component of the image, and the matrix M3 outputs the blue component of the image.

The matrices shown in FIG. 4 are not bordered by each other. By carrying out like this, an electronic device can be accommodated in the clearance gap between matrices as needed. However, as shown in FIG. 6, it is preferable to have a configuration in which four matrices are juxtaposed to form only one continuous large matrix M. This matrix therefore needs to be divided into four zones, which in principle are readable. Therefore, the read circuit associated with the matrix is divided into four components in the form of four read registers R1, R2, R3, R4 respectively associated with the four areas of the matrix M as shown in FIG. It is preferable.

Regardless of whether the configuration shown in FIG. 4 or the configuration shown in FIG. 6 is used, in both cases, to ensure that the relative positioning errors of the various optical subassemblies are taken into account, the matrix The area of the active surface is preferably larger than the maximum area of the image of the scene to be observed by at least tens of rows and tens of columns. Takes into account intrinsic positioning errors and image shifts due to the proximity of the object being photographed repeatedly, without the risk of losing part of the observed scene due to the image protruding from the matrix be able to.

For example, if it is preferable to represent an image with a resolution of 1.2 million pixels (one pixel corresponds to an image point of a specific color) composed of 1000 pixel rows and 1200 pixel columns, the present invention This is implemented by juxtaposing four different matrices. Each matrix has approximately 500 rows and 600 columns, and the pixels of these matrices have the same size and pitch as the three-color matrix pixels used in the prior art.

The lateral direction of the optical subassembly that projects the image when the side length of the matrix of various colors is half the length of the entire prior art matrix (at the same resolution and the same pixel area or sensitivity) The dimension of is also half. As a result, the focal length of the optical subassembly is half the length. Thus, the optical subassembly is not only thinner, but is placed closer to the surface of the photosensitive matrix. Accordingly, the thickness of the entire sensor is reduced. In the manufacture of small image sensors, sensor thickness is an increasingly important item. The improvement in this parameter achieved with the present invention is very important.

A batch manufacturing process is used to manufacture the sensor configured as described above. That is, not only a plurality of image sensors are manufactured on the same integrated circuit wafer, but also a wafer / plate assembly is cut into individual sensors after a plate having a plurality of optical systems is attached to the wafer. Each sensor has a silicon chip integrated with the optical system.

The optical subassembly is composed of a transparent plate and a lens formed thereon. These transparent plates can be made of glass and are molded. This subassembly can be composed of a single lens or a plurality of superimposed lenses. In the case of a plurality of lenses (convergence lens and / or diverging lens), the transparent plates are overlapped so that the gap between the lenses corresponds to the optical function to be manufactured. A fixed aperture can be similarly formed from a transparent plate covered with an opaque layer open around the optical axis of each optical subassembly, whether or not molded in the form of a lens.

FIG. 7 shows an image sensor. In this image sensor, each optical subassembly has two superimposed lenses. These two lenses are formed of molded glass plates 41 and 42 that are superimposed. Glass plates 41 and 42 are separated by a spacer plate 43 to adjust the desired vertical spacing between the lenses. One of the lenses is an opaque layer 44 (e.g., 4 locally open) to form an aperture (fixed) 45 above each matrix through which light coming from the observed scene passes. Covered with an aluminum layer).

The iris can also be formed on an additional plate positioned above the plate 42 at a specific distance from the plate 42. The stop can have a field stop function. In this case, it is preferable that the field stop has a rectangular shape (the same shape as the matrix), and the illuminance of the matrix is limited by an optical component associated with the neighboring matrix.

The sensor according to the present invention is particularly suitable when the observed scene is sufficiently distant from the image sensor, and image shifts other than essential shifts due to imperfections during manufacturing can be ignored.

A shift that cannot be ignored compared to the size of the pixel occurs because the scenes are close together (for example, the scene is placed at a distance less than N times the focal length of a matrix with N rows) The shift can be adjusted according to the proximity of the scene. The shift to be compensated is proportional to the optical subassembly spacing d and inversely proportional to the observed subject distance.

In the case of a sensor composed of four matrices, including two green matrices, the fact that two green matrices should recognize the same image if there is no intrinsic shift or shift due to subject proximity Can be used. As explained so far, intrinsic shifts can be calibrated in the factory and then taken into account in a systematic way, so when considering the explanation below, the subject It is only necessary to correct the shift due to the proximity of. Accordingly, in the following, it is assumed that the optical centers O1, O2, O3, O4 of the optical subassembly coincide with the centers C1, C2, C3, C4 of the matrices M1-M4, respectively.

If the photographed subject is at a non-infinite distance D, the image of this subject projected with two identical optical components (both centers are separated by a distance d) is relative to the center of the optical component. It is offset laterally by a distance proportional to d / D, or more precisely by a distance equal to dF / D. Here, F is a focal length.

That is, the following can be said with reference to FIG.
In FIG. 8a, a single optical assembly L with a center O is assumed, whereby an image (vector vs) of an object (vector VS) at a distance D apart is projected onto the matrix. The object VS and the image vs are aligned with the hypothetical center C of the matrix. This center is perpendicular to the optical center O.
In FIG. 8b, two optical components L1 and L4 are used instead of simple optical components. The two optical components are the same as the optical component L shown in FIG. 8a, but are offset from each other by a distance d. In other words, the centers O1 and O4 of the two optical components are offset laterally by + d / 2 and −d / 2, respectively, with respect to the center O of the single optical component. These optical components observe the same subject as in FIG. 8a (also represented by the vector VS). This subject is repeated but is centered on the two optical component groups, ie, the subject remains centered on the center O. This center O represents the central axis of the whole image sensor having two optical subassemblies. Thus, the vector VS is shifted to one side with respect to the center O1 of the first optical subassembly and shifted to the other side with respect to the center O4 of the second optical subassembly. Since D is not infinite, a parallax effect occurs.

The images vs1 and vs4 provided by the two optical assemblies are not only offset by a distance d from each other because the optical components are separated by a distance d, but each optical component sees the subject laterally. They are further offset from each other by a distance dF / D because they are not centered.

It is assumed that C1 is the center of the matrix M1 and is perpendicular to the center O1 of the optical component L1 located on the left side, and C4 is the center of the matrix M4 and the center O4 of the optical component L4 located on the left side. Assuming that is in a vertical relationship, the centers C1 and C4 are offset by a distance d. That is, C1 is offset by + d / 2 with respect to C, and C4 is offset by -d / 2 with respect to C.

The image vs1 produced by the optical component L1 is shifted to the left by the distance dF / 2D with respect to the center C1. That is, the center of the subject image does not coincide with the center C1, but is shifted to the left by dF / 2D. Similarly, the image vs4 produced by the optical component L4 is shifted to the right by the distance dF / 2D with respect to the center C4.

In the system according to the invention with four projection optics, we use the fact that two green matrices (M1 and M4) arranged along the diagonal are viewing the same scene. However, this scene is shifted towards the two matrices unless the magnification is zero.

The method of calculating the correlation between the image from the matrix M1 and the image from the continuously offset matrix M4 is a shift value that gives the optimal correlation between the images projected on the two matrices. Used to determine a shift value that represents the overall shift present. If it is assumed that there is no essential shift due to inaccurate positioning of the optical component, or if the calibration corresponding to the essential shift is subtracted from the overall shift, what remains is the proximity of the subject. The shift dF / D due to the nature. This shift is a vector oriented in the direction of the diagonal connecting the recalibrated centers of the green matrix.

The shift search algorithm is simple because the direction of the shift due to the proximity of the captured object is known. Image bands (possibly containing the observed main subject) are obtained from matrix M1 and matrix M4. These image bands are referenced with respect to the center of each matrix after taking into account calibration. Therefore, the image band is referenced with reference to the centers O1 and O4 of the optical component. This reference is the same in both matrices. That is, when the center of the image band is aligned with O1 in the matrix M1, the center of the corresponding band in the matrix M4 is aligned with O4. The image band is a sample of the entire image, since it is not necessary to establish relative relationships using the entire image.

The image bands from M1 and M4 are subtracted from each other. A representative illuminance value of the image obtained by this subtraction is determined and stored (in simple terms, this value can be the average illuminance of the image resulting from the subtraction).

To correct an image band sample, the image band sample is shifted by a distance increment along the diagonal connecting the centers of the matrices (this means a shift of one row pixel and one column pixel). Then, the other two image bands are acquired. The centers of these bands are shifted incrementally with respect to the center of the matrix. The subtraction is repeated and the illuminance of the image resulting from the subtraction is determined.

This process is repeated continuously, incrementing the relative position of the band with respect to the center of the matrix each time. Incrementation is performed in the direction in which two image bands that are farther apart (not close) along the direction of the diagonal connecting the centers of the matrices are acquired in succession. This can be understood by examining FIG. 8b. That is, when the photographed subject is close to the image sensor, the obtained image is offset and is further away from the diagonal line. Starting from a theoretical situation without offset (corresponding to the subject being located at infinity), it is advantageous to gradually search for the presence of larger shifts.

The purpose is to determine a shift that gives the minimum illuminance value to the image resulting from the subtraction. This shift indicates the dF / 2D value. This is because the minimum illuminance value is a value corresponding to a superimposable image band, and when the photographed object is separated by a distance D, each image is superimposed after being shifted by dF / 2D in the opposite direction. Because we know that it will be possible.

Once this optimal dF / 2D value is found, it is then used to do the following:
First, determine the standard green image that each green matrix provides, assuming that each green matrix is located at the exact center O of the sensor and is not offset by d / 2 from this center. Accordingly, the image of the matrix M1 is brought close to the center C1 by dF / 2D, and the image is shifted. Further, the image of the matrix M4 is brought close to the center C4 by dF / 2D, and the image is shifted.
-Second, from this value, estimate the shift required for the blue and red matrices. Since the shift due to the proximity of the photographed object is the same as in the case of the two green matrices, the red image of the matrix M2 is shifted by dF / 2D along the diagonal connecting the centers of the matrices M2 and M3, Furthermore, it is sufficient to shift the blue image of the matrix M3 by the same amount along this diagonal. Since there is a tendency to increase the distance between the optical components in the arrangement of the optical components, the shift amount dF / 2D of the red matrix and the blue matrix is in the opposite direction along the diagonal line and the direction in which the image is brought closer to the center of each matrix.

The shifts calculated for each matrix (green, blue, red) are then used to determine the recombination order that matches each image point with each point in each matrix to produce a complete color image. The

The algorithm for obtaining the optimum shift value can perform finer incrementation. For example, one pixel increment for one of the two image bands, one pixel increment for the other image band, and so on. It is also possible to wait for the next incrementation to increment one pixel at a time instead of one column and increment along a column rather than a row.

Therefore, the color image sensor with improved colorimetry as described above provides an accurate image even for a close subject.

Also, thanks to the presence of two green matrices, the apparent resolution of the sensor can be improved. For this purpose, the green matrices are very accurately positioned relative to each other by using a half-pixel relative shift between the image projected on one matrix of green and the image projected on the other matrix. (For simplicity of explanation, the essential shift due to incomplete positioning of the center of the optical component located above the matrix is considered and corrected). Therefore, a larger number of samplings are performed in the green channel. A shift is a half pixel along a row and a half pixel along a column. The projected images also need to be offset by half a pixel from each other along the diagonal.

One possible way to position two images within a half pixel is to reconstruct a normal size pixel using a matrix whose pixel dimensions and pitch are half the pixel size required by this matrix. And adding the charges of the pixels adjacent to each other. Regardless of the CCD technology or CMOS technology, reading is performed after this addition is performed for each pixel. A normal size pixel centered within a half pixel is formed in response to the smaller neighboring pixels being summed. The two matrices can have different drives so that there is effectively a half-pixel shift in between.

In CCD technology, addition (also referred to as “binning”) is performed in the readout register by four-stage transfer gate control.

In CMOS technology, the addition is performed at charge storage nodes associated with four adjacent small photodiodes. Since this node is separated from the photodiode by a transfer gate, it is possible to select which of the adjacent photodiodes recognizes the charge transferred to a given storage node by appropriately controlling these gates. . If nine adjacent photodiodes organized in rows and columns are used, the photodiodes can be grouped into four sets of adjacent photodiodes and the charge storage node placed at the center of each group. It can be divided into four groups, which constitute larger pixels that are half a pixel apart from each other along a row, along a column, or along a diagonal.

Thus, if pixels in one matrix are grouped in a certain way, the pixels in the other matrix are grouped in another way so that the shift between the two matrices is half a pixel along the diagonal. Divided.

FIG. 9 is a specific embodiment showing an octagonal photodiode having an overall area of one quarter of the photodiode provided to achieve the desired pixel pitch. The arrows indicate the discharge that is simultaneously performed from the four photodiodes to the storage node surrounded by the four photodiodes. Selecting a transfer gate that operates for a particular photodiode determines the storage node from which that photodiode is discharged. In FIG. 9a, the transfer gate is activated to group the four diodes in a specific way. On the other hand, in FIG. 9b, the transfer gate is activated to group the relevant diodes around the storage node differently. All storage nodes used in FIG. 9a (first green matrix) are offset by half a pixel with respect to the storage nodes used in FIG. 9b (second green matrix).

  Different colors if colorimetry is a more important parameter than resolution and image overlay accuracy, or if the user feels joy in observing an image of a far-planar scene that is correctly overlaid in any case The sensor can be configured to have four matrices each covered by four filters (ie, red, green, blue, cyan, each associated with one matrix). Of course, the user is restricted from viewing distant scenes because it becomes impossible to process the images resulting from the two matrices to determine and correct the parallax.

A known configuration example of a color image sensor is shown in plan view, with a series of photosensitive areas arranged in rows and columns. FIG. 2 is a sectional view showing the same sensor at a smaller scale, and shows an optical system that projects the entire image to be observed onto a matrix of photosensitive regions. FIG. 2 is a cross-sectional view of a sensor according to the present invention showing a plurality of optical subassemblies that project the same image onto a plurality of matrices of different colors. FIG. 4 is a plan view of the sensor of FIG. 3. FIG. 6 illustrates the essential shift due to relative positioning errors between the optical subassembly and the matrix. A variation is shown in which four matrices are four quadrants of a larger matrix. Fig. 4 shows an exemplary embodiment in which a plurality of lenses are provided above each matrix. It is a figure explaining the image shift resulting from the center shift | offset | difference of two optical subassemblies which observe the same object arrange | positioned in the distance of non-infinity. FIG. 6 shows a practical embodiment of a matrix where image positioning can be defined within half a pixel to improve resolution using two green matrices.

Claims (7)

A color image sensor having an optical system for projecting an image of a scene to be observed onto a series of photosensitive areas, wherein the series of photosensitive areas are manufactured on the same monolithic semiconductor chip integrated with the optical system. In the color image sensor
The optical system includes a plurality of optical subassemblies (L1, L2, L3, L4), and the series of photosensitive areas is divided into at least two matrices (M1, M2, M3, M4) that can be individually read; Each optical subassembly is designed to project the entire scene to be observed onto a respective matrix, above the various matrices, two uniform filters of different colors to pass a single color of light through each matrix A color image sensor in which (F1, F2, F3, F4) are arranged.   The series of photosensitive areas is divided into four matrices arranged in a square, and the two matrices arranged along the diagonal of the square are associated with a green filter and the remaining two arranged along the other diagonal. The image sensor according to claim 1, wherein one matrix is associated with a red filter and the other with a blue filter.   The image sensor according to claim 1, further comprising at least one calibration register in which a relative shift between the center of the optical subassembly and the center of the matrix is stored.   The matrix is larger than the image of the scene to be observed so that the relative positioning error between the optical subassembly and the matrix can be taken into account without losing image fragments. The entire image of the scene is still projected onto the four matrices, even if the center of the image is not exactly aligned with the center of the matrix. The image sensor described in 1.   Means for point-to-point correspondence of the images of the various matrices, proportional to the separation of the corresponding optical subassemblies according to the distance of the object in the scene to be observed and the distance of the object 5. The image sensor according to claim 1, further comprising means for relatively shifting the images of two adjacent matrices by a distance inversely proportional to.   Means for determining by image correlation algorithm how much the shifts the images of the two green matrices are overlaid, thereby establishing an image correspondence between the pixels of the four matrices The image sensor of claim 2, wherein the shift that must be applied to the various matrices of green, blue, and red is estimated.   2. An image sensor according to claim 1, comprising four matrices covered with four different color filters, in particular red, green, blue and cyan filters.

Images