CN111416967B - Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application - Google Patents
Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application Download PDFInfo
- Publication number
- CN111416967B CN111416967B CN202010205091.8A CN202010205091A CN111416967B CN 111416967 B CN111416967 B CN 111416967B CN 202010205091 A CN202010205091 A CN 202010205091A CN 111416967 B CN111416967 B CN 111416967B
- Authority
- CN
- China
- Prior art keywords
- color filter
- image
- color
- sensor
- elements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/843—Demosaicing, e.g. interpolating colour pixel values
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Optical Filters (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
The application provides a color image sensing device, a manufacturing method thereof, an image processing device and application. A color image sensing device comprises an image sensor and an associated primary color filter array, wherein the image sensor comprises a sensor array, the sensor elements in the sensor array correspond to color filter elements in the primary color filter array one by one, one color filter element corresponds to one pixel, two pixels or three pixels and is respectively called a single color filter element, a double color filter element and a triple color filter element, correspondingly, the single sensor element, the double sensor element and the triple sensor element exist, wherein all the primary color filter element arrays form the color filter element array, the number of at least one of the double color filter element and the triple color filter element is not zero, and correspondingly, the number of at least one of the double sensor element and the triple sensor element is not zero. The one-to-one correspondence among the traditional color filter element array, the image sensing element array and the color image pixel array is broken through, so that the number of the sensor elements is reduced under the condition that the image pixels are not changed, and the area of the sensor elements is relatively enlarged.
Description
Technical Field
The present invention relates generally to color image sensing technology, and more particularly to a color image sensing device and an apparatus for acquiring a digital color image, which can be used as a digital color image and video acquisition apparatus for various mobile phones, tablet computers, notebook computers, digital cameras, digital video cameras, etc., and also to a rendering and arranging method of a color filter array of a color image sensing device, and an image reconstruction method.
Background
Digital color image sensing devices are the most critical components in digital image devices, such as digital cameras and digital video cameras, including mobile phones and tablet computers, and are used to convert optical images focused by a lens into digital color images. The color image sensing device is composed of a color filter array and an image sensor. The Image Sensor (Image Sensor) functions to convert the brightness of visible light irradiated onto the Sensor into an electric signal, and digitize it in a subsequent process to generate a gray-scale (black-and-white) Image. In order to obtain Color information in nature to generate a Color image, a method of covering a Color Filter Array (CFA) on the surface of an image sensor is generally adopted, and a Color image sensing device is formed by the Color Filter Array and the image sensor.
The digital image sensor functions to digitize the intensity of visible light impinging on the sensor, producing a gray-scale (black and white) image. Since a digital image is a pixel array formed by arranging a plurality of pixels, the image sensor is a sensor cell array formed by arranging a plurality of sensor cells, wherein each sensor cell independently digitizes the intensity (brightness) of the received light to generate one pixel in the digital image. The color image sensing device is characterized in that a color filter element array covers the surface of an image sensor and corresponds to the sensor element array on the image sensor one by one, so that the image sensor can digitize light intensity (brightness) of different colors in visible light, and a color image is generated after reconstruction of computing equipment. Fig. 11 is a schematic diagram showing a structural configuration of the color image sensing apparatus, where reference numeral 1 denotes a color filter array, reference numeral 2 denotes an image sensor, and the image sensor is configured by an arrangement of sensor elements, where R ═ Red denotes a Red filter element, G ═ Green denotes a Green filter element, B ═ Blue denotes a Blue filter element, T ═ Transparency filter element denotes a transparent filter element, and the filter elements and the sensor elements correspond to each other one to one.
Color digital image devices can be classified into three types according to the number of color image sensors inside the devices: (1) the color image sensing device is composed of an image sensor and a three-color filtering element array covering the image sensor, wherein one third of color image signals are obtained, and the color image is reconstructed after the signals are processed by computing equipment. The color image sensor based on the single image sensor has the advantages of small sensor area, low price, mature manufacturing method and the like, and is widely applied to various digital image devices such as mobile phones, tablet computers, digital cameras, digital video cameras and the like along with the development of computing devices. (2) A color digital image device with multiple image sensors, such as a 3CCD digital camera, a 3CMOS digital camera, etc., has a light splitting polygon prism inside, which splits the externally incident visible light into three beams of red, green and blue, which are received by three independent image sensors (CCD or CMOS image sensors) respectively and quantized into color digital images. The 3CCD or 3CMOS color image sensor has no color filter array, but needs an optical spectroscope, has large volume and complex manufacture, and is only used in some medium and large digital cameras at present. (3) A color digital image device of a stacked multi-image sensor, i.e., an X3 color image sensor (japanese Sigma corporation patent), employs a three-layer three-color structure, i.e., three-layer overlapped image sensors, which are respectively a blue, green, and red image sensor from top to bottom. The blue part of the incident light from the outside is firstly absorbed and digitalized by the first layer of blue image sensor, the second layer of green image sensor absorbs the green part of the incident light and digitalizes the green part, and the third layer of red image sensor absorbs the rest light and digitalizes the rest light, thereby generating a color digital image. In the X3 color image sensor, due to the lamination of the sensors, the upper layer sensor and the circuit thereof have certain attenuation effect on light, so that the light which can be received by the lower layer sensor is greatly reduced. In order to solve this problem, such image sensors increase the signal-to-noise ratio by increasing the area of the sensor and the area of the sensor elements included in the sensor, so that the entire camera is large in size, and is generally used in a medium-sized digital camera such as a single lens reflex camera.
Technical research on existing color image sensing devices has also focused on the color rendering and arrangement of color filter cells, for example, the most common color filter cell array in a color image sensor is a Bayer color filter cell array, which is formed by repeatedly arranging 2 × 2 RGGB (red green blue) color filter cells.
The color filter element array has two basic forming modes, one is that color filter elements of various colors are randomly arranged to form the color filter element array, and the other is that a color filter element group is formed by firstly forming color filter elements of different colors, and then the color filter element group is repeatedly arranged to form the color filter element array. There are related patents for both arrangements, and there are many patents for color filter sets, and the main existing color filter set technologies are listed in table 1.
Table 1: color and arrangement of color filter element group in color filter element array
The color filter element array in the prior art has the following main characteristics:
1) the color filter array consists of three-color or four-color filter elements which are randomly arranged or repeatedly arranged in groups, and the color filter elements have various forms such as 2x2, 2x4, 3x3, 4x4 and the like;
2) the color filter elements in the color filter element group have a determined arrangement mode;
3) each color digital sensing device comprises a color filter element array and an image sensor, and each sensing element on the sensor only collects signals of one of three colors or four colors;
4) of particular importance is that the color filter elements in the color filter element array are in one-to-one correspondence with each of the sensor elements in the image sensor array, each being a square, and that one color filter element and corresponding sensor element combination corresponds to one pixel.
Although the color digital image sensing device formed by the color filter array and the image sensor changes the image sensing device from a gray image sensing device to a color digital image sensing device, the color filter array filters 2/3 light, only 1/3 light actually reaches the image sensor for imaging, and the computing device reconstructs a color image based on 1/3 sampling information of the color image sensing device by adopting a certain algorithm.
The color sampling of 1/3 reduces the signal-to-noise ratio of the color image sensor due to the inevitable random sampling noise per sensor element, while introducing independent random noise in the sampling of the color of each pixel. In the traditional single-lens reflex camera, the micro single-lens camera, the professional camera and other medium and large cameras, each sensing element obtains more light rays by increasing the area of the sensor and correspondingly increasing the lens of the camera, and the signal-to-noise ratio of imaging is improved to be relieved. However, with the popularization of mobile devices such as mobile phones, in order to reduce the size and weight of a color image imaging module, the area of an image sensor in a color image sensing device used in the mobile phone is very limited, the number of pixels is increasing, and the area of a single pixel in the image sensor is reducing. Therefore, when the light of the color image sensing device is insufficient, the signal-to-noise ratio is low, and the quality of the color image acquired by the color image sensing device when the light is insufficient is seriously influenced. The post-processing color digital image processing and enhancement algorithm (as shown in fig. 10A) has difficulty in suppressing random color noise generated during the imaging stage.
The above problem cannot be solved by increasing the area of the image sensor and increasing the lens accordingly as in the conventional camera. The Huache-et al mobile phone manufacturer improves the signal-to-noise ratio of the image by adding an auxiliary gray-scale camera, thereby partially alleviating the above problems. For example, in cellular phones of models P9 and P9Plus, and in cellular phones of models Cool1, a dual-camera scheme of a color camera and a grayscale camera is introduced, wherein the color camera is used for obtaining the color of an object, and the grayscale camera is used for obtaining the bright and dark details of the object. The color image and the gray scale image are then fused by color digital image processing and enhancement algorithms (see fig. 10B), so that the color image taken in dark light can also make the object have rich details. Thereby suppressing the overall noise of the color image and improving the quality of the color image. However, since the color image sensing apparatus of the color camera itself is not changed in the above scheme, the signal-to-noise ratio of the color image sensing apparatus is not improved. Because the red, green and blue noises on the color image sensing device are random and independent from each other, the scheme of the double cameras can enhance the details of the image, but still can not remove the noise from the color image sensing device in the color image, namely the color noise in the color image.
Besides the above-mentioned color + black and white dual-camera scheme, there are also dual-camera schemes adopted by mobile phones, such as a dual-camera-based stereo camera and a dual-camera-based optical zoom. For example, the dual camera of iPhone 7Plus of apple company adopts two identical color digital image sensing devices but different lenses, i.e. the lenses of the dual camera are wide-angle + telephoto lenses, for realizing the integration of optical zooming and digital zooming. These dual cameras are based on the combination of the existing color camera and color camera or grayscale camera, and then algorithm software is used to obtain multiple functions such as color image enhancement, fusion of optical zoom and digital zoom, stereoscopic vision, etc., but like the above dual-camera scheme, there is no innovation on the color image sensing device itself.
Disclosure of Invention
In order to make the present invention better understood by those skilled in the art, the operation principle of a general color image sensing apparatus will be briefly described. As shown in fig. 11, the color image sensing device is composed of two parts, namely, an image sensor and a color filter array, wherein the color filter array has a plurality of color rendering modes and arrangements, and the common feature is that the color filter array at least includes color filters of three primary colors (red, green, blue, or cyan, yellow, magenta), which is called a full-color filter array (panchromatic-CFA). In a color sensing device (also commonly referred to as a color sensor or a color image sensor by those skilled in the art), the color filter elements in the panchromatic color filter element array correspond to the sensing elements in the image sensor one by one, and are repeatedly arranged to form a multi-pixel color image sensing device.
The traditional approach for improving the signal-to-noise ratio of the color image is to improve the manufacturing method or process of the image sensor, so as to reduce the sampling noise of the sensor; or the area of each sensing element in the image sensor is increased, and two realization ways can be provided, namely, the area of the image sensor is increased, so that the area of each sensing element is increased; and secondly, under the condition that the area of the image sensor is fixed, the number of pixels of the image sensor is reduced, so that the area of each sensing element is increased. However, increasing the area of the sensor requires a corresponding increase in the size of the lens, which increases the size of the entire color camera module, contrary to the miniaturization requirement of modern mobile devices; the reduction of the pixel count of the color image sensor is contrary to the development trend and market demand of the current color image sensor. Therefore, the implementation approach for improving the signal-to-noise ratio of the color image is not feasible under the current image sensor manufacturing process under the trends of miniaturization and high pixelation of mobile equipment.
In order to solve the problems, the invention is based on the theoretical result of random imaging proposed by long-term research, and revolutionarily breaks through the tradition that one color filter element and one sensor element pair which are consistently adopted for decades correspond to one pixel in a color image and the sensor elements are square, namely, the sensor element array corresponds to the image pixel array one by one. The invention adopts a method that one color filter element and one image sensor element correspond to a plurality of pixels, thereby reducing the number of the sensor elements under the condition of keeping the total amount of the image pixels unchanged, equivalently increasing the area of each sensor element and improving the signal-to-noise ratio of the color image sensor.
According to an aspect of the present invention, an image sensing apparatus includes one or two image sensors and associated one or two arrays of primary color filter elements, the image sensors including an array of sensor elements, the sensor elements in the array of sensor elements corresponding to the color filter elements in the array of primary color filter elements one to one, one color filter element corresponding to one pixel, two pixels, or three pixels, wherein one color filter element corresponds to a called primary color filter element of one pixel, one color filter element corresponds to a called dual color filter element of two pixels, one color filter element corresponds to a called tri color filter element of three pixels, the dual color filter element has an area twice that of the single color filter element, and the three color filter elements has an area three times that of the single color filter element, correspondingly, one sensor element corresponds to a called primary sensor element of one pixel, one sensor element corresponds to a called dual sensor element of two pixels, and one sensor element corresponds to a tri color filter element of three pixels, the area of the double sensing element is twice that of the single sensing element, the area of the three sensing elements is three times that of the single sensing element, wherein at least one element color filter element array comprises color filter elements of at least two primary colors, and all element color filter element arrays form a full color filter element array with all primary colors, wherein the number of at least one of the double color filter elements and the three color filter elements is not zero, and correspondingly, the number of at least one of the double sensing element and the three sensing elements is not zero.
Optionally, the image sensing device comprises two image sensors and two primary color filter arrays, the total primary colors being three primary colors or four primary colors, the four primary colors being red, green, blue and colorless, at least one of the two primary color filter arrays comprises filter elements of at least two primary colors, and all the two primary color filter arrays constitute a full color filter array with the total primary colors.
Optionally, the color filter array includes two or three of single color filter elements, double color filter elements and triple color filter elements, the sensor element array includes two or three of corresponding single sensor elements, double sensor elements and triple sensor elements, a ratio between three color filter elements in at least one element color filter element array meets a predetermined ratio relationship, according to the predetermined ratio relationship, two or three colors of single color filter elements, double color filter elements and triple color filter elements are arranged randomly or satisfying a certain constraint condition, a random mixed triple color filter element array is formed, and the number of at least one of the double color filter elements and the triple color filter elements is not zero.
Optionally, the total primary colors are three primary colors RGB, and the three primary colors R: G: B ═ 1: 2: 1, wherein the red color filter element, the blue color filter element and the corresponding sensor element are a single color filter element and a single sensor element, and the green color filter element and the corresponding sensor element are a double color filter element and a double sensor element.
Optionally, the overall primary color is red, green, blue, white or a transparent color.
Optionally, only dual color filter elements are included in the color filter element array, and correspondingly, only corresponding dual sensor elements are included in the sensor element array.
Optionally, only three color filter elements are included in the color filter element array, and correspondingly, only three corresponding sensor elements are included in the sensor element array.
Optionally, the three color filter elements and the three sensor elements are L-shaped.
Optionally, two image sensors and associated two arrays of metamer color filter cells; the first color filter element array consists of a first primary color filter element and a second primary color filter element; the second color filter element array is composed of color filter elements of a third primary color, and the color filter elements of the first primary color, the second primary color and the third primary color are double color filter elements.
Optionally, two image sensors and associated two arrays of metamer color filter cells; the first color filter element array consists of a first primary color filter element and a second primary color filter element; the second color filter element array is composed of color filter elements of a third primary color and transparent color filter elements.
Optionally, the color filter of the first primary color, the color filter of the second primary color, the color filter of the third primary color, and the transparent color filter are all dual color filters.
According to another aspect of the present invention, there is provided a method of manufacturing an image sensing apparatus, including: determining the proportion between the three primary color filter elements; randomly determining the number of single color filter elements, double color filter elements and triple color filter elements of the three colors and accordingly determining the number of corresponding single sensor elements, double sensor elements and triple sensor elements based on the proportion; manufacturing single color filter elements, double color filter elements and triple color filter elements of the determined number of three colors, and manufacturing corresponding single sensor elements, double sensor elements and triple sensor elements; and forming a random mixed three-color image sensor by manufacturing the determined number of single color filter elements, double color filter elements and three color filter elements of the three colors and corresponding single sensor elements, double sensor elements and three sensor elements.
According to another aspect of the present invention, an image processing apparatus includes the above image sensing apparatus and an image processor, wherein the image processor obtains each corresponding sub-color image based on a signal sampled by each image sensor, and reconstructs to obtain a final full-color image based on the each corresponding sub-color image, wherein in a case where a corresponding meta-color filter array of one image sensor includes a plurality of color filter elements of primary colors, the image processor obtains a monochromatic sub-color image corresponding to each of the plurality of primary colors based on the signal sampled by the image sensor; under the condition that an element color filter element array corresponding to one image sensor comprises a single primary color filter element, an image processor obtains a single-color sub-color image corresponding to a single primary color based on a signal sampled by the image sensor; under the condition that a primary color filter element array corresponding to one image sensor comprises colorless filter elements, a corresponding gray level image is also obtained based on a signal sampled by the image sensor; and the image processor reconstructs the final color image and gray image with high pixel quantity by adopting the sub-color image and the sub-gray image with low pixel total quantity, which are obtained by adopting the double color filtering elements, the three color filtering elements and the corresponding sensing elements, by adopting a demosaicing, noise reduction and super-pixel method based on the single sub-color image corresponding to each image sensor.
According to another aspect of the present invention, there is provided a computing apparatus including the image processing device described above.
Optionally, the computing device is one of a smartphone, a tablet computer, a notebook computer, a digital camera, and a digital video camera.
According to another aspect of the present invention, there is provided an image reconstructing method for reconstructing a color image using signals obtained by the above image sensing apparatus, including obtaining respective corresponding sub-color images based on signals sampled by respective image sensors, and reconstructing to obtain a final full-color image based on the respective corresponding sub-color images, wherein in a case where a corresponding primary color filter cell array of one image sensor includes a plurality of filter cells of primary colors, a monochromatic sub-color image corresponding to each of the plurality of primary colors is obtained based on the signals sampled by the image sensor; under the condition that an element color filter element array corresponding to one image sensor comprises a single primary color filter element, obtaining a single-color sub-color image corresponding to a single primary color based on a signal sampled by the image sensor; under the condition that a primary color filter element array corresponding to one image sensor comprises colorless filter elements, a corresponding gray level image is also obtained based on a signal sampled by the image sensor; and reconstructing the sub-color images and the sub-gray level images with low pixel total amount obtained by sampling the double color filtering elements, the three color filtering elements and the corresponding sensing elements by adopting a demosaicing, noise reduction and super-pixel method based on the single sub-color images corresponding to the image sensors and the gray level images under the condition that the gray level images exist.
According to the color image sensing device comprising the special color filter element and the sensing element, the signal-to-noise ratio of the color image sensing device can be remarkably improved, and meanwhile, the number of the sensor modules is increased, the volume of a single sensor and a lens is not increased, the manufacturing difficulty and the cost of the single sensor are not increased, so that the color image sensing device is easy to use on mobile devices such as a mobile phone and the like; according to the color image sensing device with more than two image sensors and the corresponding reconstruction algorithm, the anti-noise capability of the color image sensor is improved, the accuracy of color restoration is improved, and therefore the performance of the color image sensing device is improved.
Drawings
These and/or other aspects and advantages of the present invention will become more apparent and more readily appreciated from the following detailed description of the embodiments of the invention, taken in conjunction with the accompanying drawings of which:
FIG. 1A shows a schematic block diagram of an image sensing device according to one embodiment of the present invention; FIG. 1B shows a random three-color-mixed sensor cell array corresponding to the random three-color-mixed filter cell array of FIG. 1A;
fig. 2A and 2C illustrate a color mixing triple filter array formed by arranging color filter cells, double color filter cells and triple color filter cells of three colors according to an embodiment of the present invention; FIGS. 2B and 2D illustrate a three-color mixing sensor element array corresponding to the three-color mixing filter element array of FIGS. 2A and 2C, according to an embodiment of the invention;
fig. 3A illustrates a random mixed dual color filter cell array formed using randomly generated single and dual color filters of three colors according to an embodiment of the present invention, and fig. 3B illustrates a random mixed dual sensor cell array corresponding to the random mixed dual color filter cell array illustrated in fig. 3A according to an embodiment of the present invention.
Fig. 4A and 4C illustrate a color filter array including dual color filters according to an embodiment of the present invention, and fig. 4B and 4D illustrate a sensor cell array including dual sensor cells corresponding to the color filter array of fig. 4A and 4C according to an embodiment of the present invention.
Fig. 5A and 5C illustrate a color filter array including dual color filters according to an embodiment of the present invention, and fig. 5B and 5D illustrate a sensor cell array including dual sensor cells corresponding to the color filter array of fig. 5A and 5C according to an embodiment of the present invention.
Fig. 6A shows a color filter array composed of a color filter array of mixed double color filters according to an embodiment of the present invention, and fig. 6B shows a sensor cell array composed of mixed double sensor cells corresponding to the color filter array of fig. 6A according to an embodiment of the present invention.
Fig. 7A illustrates a color filter array composed of three color filters according to an embodiment of the present invention, and fig. 7B illustrates a sensor cell array composed of three sensor cells corresponding to the color filter array of fig. 7A according to an embodiment of the present invention.
FIG. 8A shows a primary color filter cell array according to an embodiment of the present invention, which includes two color filter cells, while FIG. 8C shows a primary color filter cell array according to another embodiment of the present invention, which includes only one color filter cell; fig. 8B and 8D show that the dual image sensor of the color image sensing apparatus includes two image sensors, in which the monochrome image sensing apparatus composed of the dual color filter array shown in fig. 8C plus the image sensor shown in fig. 8D is full-sampled, and the dual color image sensing apparatus composed of the dual color filter array shown in fig. 8A plus the image sensor shown in fig. 8B is 1/2-sampled for two colors.
FIG. 9A illustrates a meta-color filter cell array containing one color and transparent filter cells according to an embodiment of the invention; fig. 9B illustrates that the dual image sensor of the color image sensor includes two image sensors.
Fig. 10A and 10B illustrate a color digital image processing and enhancing algorithm in a later stage of a conventional color image sensing apparatus.
Fig. 11 is a schematic view showing a structural composition of a conventional color image sensing apparatus.
Detailed Description
In order that those skilled in the art will better understand the present invention, the following detailed description of the invention is provided in conjunction with the accompanying drawings and the detailed description of the invention.
In this context, a color image sensing device is composed of an image sensor composed of sensing cells and a color filter cell array composed of an arrangement of color filter cells.
An image sensing apparatus according to a first embodiment of the present invention
An image sensing apparatus according to a first embodiment of the present invention includes one or two image sensors and associated one or two primary color filter cell arrays, the image sensors including sensor cell arrays, the sensor cells in the sensor cell arrays having one-to-one correspondence in hardware with the color filter cells in the primary color filter cell arrays.
One color filter element corresponds to one pixel, two pixels or three pixels, wherein one color filter element corresponds to a single color filter element of one pixel, one color filter element corresponds to a double color filter element of two pixels, one color filter element corresponds to a triple color filter element of three pixels, the area of the double color filter element is twice that of the single color filter element, the area of the triple color filter element is three times that of the single color filter element, correspondingly, one sensor element corresponds to a single sensor element of one pixel, one sensor element corresponds to a double sensor element of two pixels, one sensor element corresponds to a triple sensor element of three pixels, the area of the double sensor element is two times that of the single sensor element, and the area of the triple sensor element is three times that of the single sensor element.
The at least one primary color filter element array comprises at least two primary color filter elements, and all the primary color filter element arrays form a full-color filter element array with all primary colors, wherein the number of at least one of the double color filter elements and the triple color filter elements is not zero, and correspondingly, the number of at least one of the double sensor elements and the triple sensor elements is not zero.
Fig. 1A shows a schematic composition diagram of an image sensing apparatus according to an embodiment of the present invention.
Specifically, the ratio between the three primary color filter elements in the color filter element array may be first set, and the single, double, and triple color filter elements of the three colors are randomly generated to form a random mixed three-color filter element array (fig. 1A). The random three-color mixing sensor element array (shown in figure 1B) corresponds to the random three-color mixing filter element array, and the random three-color mixing sensor element array and the random three-color mixing filter element array are in one-to-one correspondence to form the random three-color mixing image sensing equipment.
The above-described embodiment of the present invention, a random three color sensor, is a general model of the hybrid array color sensor of the present invention. An infinite number of hybrid array color image sensing device embodiments can be derived based on the generic hybrid array color image sensing device model of the present invention, and the embodiments described are not intended to be limiting of the present invention. In addition, according to the random mixed array color image sensing device embodiment, a plurality of colors, such as red, green, blue, white or transparent colors, can be blended to form the random four-color sensing device embodiment.
Second, an image sensing apparatus according to a second embodiment of the present invention
Embodiment two is an instantiation of embodiment one generic model. In this embodiment, the ratio of the three primary color filter elements in the color filter element array is set to R: G: B ═ 2:3:1, and the three color filter elements, the double filter element, and the triple filter element are arranged to form a mixed three color filter element array (fig. 2A, fig. 2C). And three mixed sensor element arrays (shown in fig. 2B and 2D) corresponding to the three mixed color filter element arrays correspond to one another to form the three mixed color image sensing equipment.
Image sensing apparatus according to third embodiment of the present invention
The third embodiment of the present invention is an instantiation of the first embodiment of the generic model. In the general model of the third embodiment, only a single color filter and a double color filter are included in the color filter array, thereby constructing a random mixed double array. The ratio of three primary colors is set in the color filter array, and single color filters and double color filters of three colors are randomly generated to form a random mixed double color filter array (figure 3A) and a corresponding random mixed double sensor array (figure 3B), and the random mixed double color image sensing device is formed by the random mixed double color filter array and the random mixed double sensor array which are in one-to-one correspondence. Based on the general mixed double color image sensing equipment model of the third embodiment of the invention, more examples of double color filter elements and double sensing element color image sensing equipment can be deduced. Thus, the described embodiments are not an enumeration of the present invention. In addition, according to the above-described random double color image sensing device embodiments, embodiments in which multiple colors, such as red, green, blue, white, or transparent colors, are fused to form a random double four-color sensing device array may be employed.
Fourth, an image sensing apparatus according to a fourth embodiment of the present invention
The fourth embodiment of the invention is an instantiation of the embodiment three-use model. In example four, the ratio between the three primary colors is set as R: G: B ═ 1: 2: and 1, the color filter element array only has double color filter elements, and the sensor element array only has double sensor elements, which are fused into a double color filter element and a double sensor element pair. Both are rectangular in shape. Fig. 4A and 4C show a color filter array composed of dual color filters, and fig. 4B and 4D show an image sensor composed of dual sensor cells.
The fourth embodiment of the color image sensing device with dual color filters and dual sensor elements is characterized in that the ratio of three primary colors R: G: B: 1: 2: 1. this embodiment of the dual color sensor array is not an enumeration of the present invention, and the hybrid color sensor device formed by the fusion of the dual color filter elements and the dual sensor elements may have various permutations and combinations. According to the above-mentioned dual color filter cell array and dual sensor cell array embodiments, a plurality of colors, such as red, green, blue, white or transparent colors, may be blended to form an embodiment of a dual four-color sensor device.
Fifth, an image sensing apparatus according to a fifth embodiment of the present invention
The fifth embodiment of the present invention is an instantiation of the third embodiment of the present invention. In example five, the ratio between the three primary colors is set as R: G: B ═ 1: 1: and 1, only double color filter elements and double sensor elements are arranged in the color filter element array and the sensor element array, and are fused into a double color filter element and double sensor element pair. The area of the double color filter elements is one time of that of the single color filter elements, and the area of the double sensor elements is one time of that of the single sensor elements. Both are rectangular in shape. Fig. 5A and 5C show a color filter array composed of dual color filters, and fig. 5B and 5D show an image sensor composed of dual sensor cells. The dual color sensor array embodiment is not an enumeration of the present invention, and the hybrid color sensing device formed by fusing the dual color filter elements and the dual sensing elements can be arranged and combined in various ways. According to the above-mentioned dual color filter cell array and dual sensor cell array embodiments, a plurality of colors, such as red, green, blue, white or transparent colors, may be blended to form an embodiment of a dual four-color sensor device.
Sixth, an image sensing apparatus according to a sixth embodiment of the present invention
The sixth embodiment of the present invention is an instantiation of the embodiment three-use model. In example six, the ratio between the three primary colors is set as R: G: B ═ 1: 2: 1, the color filter element array and the sensor element array comprise double color filter elements, single color filter elements, double sensor elements and single sensor elements. The area of the double color filter elements is one time of the color filter elements, and the area of the double sensing elements is one time of the sensing elements. The double color filter elements and the double sensor elements are in the same shape and are rectangular. Fig. 6A shows a color filter array composed of mixed dual color filters, and fig. 6B shows an image sensor composed of dual sensor cells.
The above three embodiments (four, five, and six embodiments) of the hybrid dual color image sensor are not intended to limit the present invention, and the hybrid color image sensing apparatus based on dual color filter cells, dual color sensor cells, and conventional single color filter cells and single color sensor cells can be arranged in various combinations. According to the embodiments of the mixed dual color filter cell array and the mixed dual sensor cell array, a plurality of colors, such as red, green, blue, white or transparent colors, can be blended to form an embodiment of the mixed dual four-color sensing device.
Seventh, image sensing apparatus according to seventh embodiment of the present invention
The seventh embodiment of the present invention is an instantiation of the first embodiment of the generic model. In example seven, the ratio between the three primary colors is set as R: G: B ═ 1: 2: 1, only three color filter elements are arranged in the color filter element array and the sensing element array, and the three color filter elements and the three sensing element pairs are fused into a whole. The area of the three color filter elements is three times that of the color filter elements, and the area of the three sensing elements is three times that of the sensing elements. Both are rectangular in shape. Fig. 7A shows a color filter array composed of three color filters, and fig. 7B shows an image sensor composed of three sensor cells.
The seventh embodiment of the color image sensor with three color filters and three sensor elements is characterized in that the ratio of the three primary colors R: G: B ═ 1: 2: 1. the embodiment of the three-color sensor array is not an enumeration of the invention, and the three-color sensor formed by fusing the three color filter elements and the three sensor elements can be arranged and combined in various ways. According to the embodiments of the three-color filter cell array and the three-sensor cell array, a plurality of colors, such as red, green, blue, white or transparent colors, can be blended to form an embodiment of the four-color sensor device.
Eighth, image sensing apparatus according to eighth embodiment of the present invention
In an eighth embodiment of the present invention, the color filter array of the dual-sensor color image sensing device adopts a full-color filter array with three primary colors of red, green and blue, which is composed of two dual-color filter arrays as shown in fig. 8A and 8C, and the fig. 8A primary color filter array includes two color filter elements, while the fig. 8C primary color filter array only includes one color filter element; the dual image sensors of the color image sensing apparatus include two image sensors (fig. 8B, fig. 8D). In this embodiment, the monochrome image sensing device composed of the double color filter array plus the image sensor (fig. 8D) shown in fig. 8C is full-sampling, while the two colors in the double color image sensing device composed of the double color filter array plus the image sensor (fig. 8B) shown in fig. 8A are 1/2 sampling.
It should be noted that the color filter array shown in fig. 8C is a full sample because it only contains one color, and therefore, the color filter degraded into green in the color filter array can be located in the lens or covered on the sensor.
The dual-sensor color image sensor of the embodiment eight dual color filter elements and the dual sensor elements is a general model of the dual-sensor color image sensor of the invention. More examples of the dual-sensor color image sensor can be derived based on the dual-sensor color image sensor general model. Thus, the embodiments described are not an enumeration of the present invention, and various permutations and combinations may be derived based on the general model of a dual-sensor color image sensor. In addition, according to the present embodiment, a plurality of colors, such as red, green, blue, white, or transparent colors, may be blended to constitute an embodiment of the dual four-color dual-sensor color image sensing apparatus.
Ninth, image sensing apparatus according to ninth embodiment of the present invention
In a ninth embodiment of the present invention, the color filter array of the dual-sensor color image sensor employs a full-color filter array having four primary colors, such as red, green, blue, and transparent, which is composed of two dual-color filter arrays as shown in fig. 8A and 9A, and the fig. 8A primary color filter array includes two color filters, and the fig. 9A primary color filter array includes one color filter and a transparent color filter; fig. 9B shows that the dual image sensor of the color image sensor includes two image sensors (similar to fig. 8B). In this embodiment, the monochrome image sensing device composed of the double color filter array and the image sensor shown in fig. 9A samples a monochrome image, and the transparent color filter array samples a grayscale image, whereas the two colors in the double color filter array and the image sensor shown in fig. 8A are 1/2 samples. Therefore, the dual sensor color sensing device samples simultaneously to obtain a color image and a gray image.
The dual-sensor color image sensor with nine dual color filters and two sensor elements in the embodiment of the invention is a general model of the dual-sensor color image sensor in the invention. More examples of the dual-sensor color image sensor can be derived based on the dual-sensor color image sensor general model. Thus, the described embodiments are not an enumeration of the present invention, and various permutations and combinations may be derived based on a general model of a dual sensor color image sensing device.
Two full-pixel monochromatic images can be reconstructed by adopting algorithms such as compressed sensing, interpolation, deep learning and the like. And an image obtained by reconstructing data obtained by the sensing element corresponding to the colorless color filter element is a gray image. The color digital image is output after algorithms such as noise reduction, sharpening, image enhancement, fusion processing of a gray image and a color image and the like of four images of red, green, blue, gray and the like.
The color image sensing device (double camera) formed by the arrangement and combination of the multiple color filtering elements (single color filtering element, double color filtering element and triple color filtering element), the multiple sensing elements (single sensing element, double sensing element and triple sensing element), the double sensor and the like can obviously improve the signal to noise ratio of the color image sensing device, simultaneously reduce the number of the sensing elements, do not increase the volumes of the sensor and a lens, and reduce the manufacturing difficulty and the cost of the sensor, so that the color image sensing device is easy to use on mobile devices such as a mobile phone and the like.
Embodiments one to nine of the present invention provide a color image sensing apparatus and a method for acquiring a color digital image signal, the color image sensing apparatus being composed of a plurality of color filter cells (single color filter cell, double color filter cell, triple color filter cell), a plurality of sensor cells (single sensor cell, double sensor cell, triple sensor cell), one or two image sensors and an associated one or two element color filter cell array, and the like. In one example, an image sensor, a meta-color filter array is covered on the surface of the image sensor, and the meta-color filter array corresponds to the sensing elements on the image sensor one by one, and the sensing elements comprise double sensing elements or three sensing elements. The color image sensor device formed by the array of the multiple color filter elements and the corresponding image sensor, and the lens and other components form a color image camera, one image sensor and the corresponding color filter element array are assisted by other devices to form a camera, and the double cameras are formed by configuring one camera, and can be applied to any electronic device (such as a smart phone). The computing device equipped with the color image camera reconstructs RGB tristimulus values of the color image based on the output of the dual image sensor using a certain algorithm, such as an interpolation algorithm, a compressive sensing algorithm, a machine learning algorithm, and the like, to form a color image (fig. 11). The color image sensing equipment comprising the double color filter elements and the double sensing elements and the corresponding reconstruction algorithm improve the anti-noise capability of the color image sensing equipment and the accuracy of color restoration, thereby improving the performance of the color image sensing equipment.
Summarizing, the invention has at least the following innovation points:
(1) a new sensor element form and sensor element array are provided: the sensor elements corresponding to two pixels (two adjacent pixels), so-called double sensor elements herein, and the sensor elements corresponding to three pixels (three adjacent pixels), so-called double sensor elements and triple sensor elements herein, look like the conventional two adjacent sensor elements are merged into one sensor element, and the conventional three adjacent sensor elements are merged into one sensor element; thus, unlike a conventional sensor cell array (each sensor cell corresponds to one pixel), in the sensor cell array of the present invention, there is at least one sensor cell corresponding to two adjacent pixels or three pixels;
(2) new filter cell forms and arrays of filter cells are proposed: the color filter elements corresponding to two pixels (two adjacent pixels), so-called double sensor elements herein, and the color filter elements corresponding to three pixels (three adjacent pixels), so-called double filter elements and triple filter elements herein, look like combining two conventional adjacent color filter elements into one color filter element, and three conventional adjacent color filter elements into one color filter element; thus, unlike a conventional color filter array (each color filter corresponds to one pixel), in the color filter array of the present invention, there is at least one color filter corresponding to two adjacent pixels or three adjacent pixels;
(3) an image sensing device is proposed which is composed of two sets of color filter arrays and sensor cell arrays, each set of color filter arrays and the following sensor cell arrays constituting an image sensor, the image sensing device having a dual image sensor, wherein the most typical image sensor can be manufactured as follows: (3.1) two adjacent color filtering elements are combined into a double color filtering element, and the color filtering element array is formed by arranging the double color filtering elements or mixing and arranging the double color filtering elements and the color filtering elements and comprises a plurality of colors; (3.2) correspondingly, the two sensing elements are combined into a double sensing element, and the sensing element array is formed by arranging the double sensing elements or mixing and arranging the double sensing elements and the sensing elements; (3.3) the image sensing device comprises two color filter arrays, two image sensors (or two sensor arrays), wherein each color filter array and one sensor array form a group, wherein the two color filter arrays have different colors, and the colors of the two color filter arrays are combined to form a full color filter array (3.4).
(4) A method for reconstructing a color image by using algorithms such as demosaicing, noise reduction, superpixel and the like is provided. The super-pixel algorithm comprises a two-super-pixel and three-super-pixel algorithm and is used for reconstructing sub-color images and sub-gray images of 1/2 and 1/3 pixel total amount obtained by sampling double color filtering elements, three color filtering elements and corresponding sensing elements to obtain final color images and gray images of full pixels.
The color image sensing device is a key module of mobile devices such as mobile phones, and the unit size of an image sensor in the current color image sensing device is close to the limit. The invention is a major breakthrough in the field of color image sensing equipment, and breaks through the one-to-one correspondence among the traditional color filter element array, the image sensing element array and the color image pixel array in the aspects of color signal acquisition, color image reconstruction and the like, so that the color filter element array and the image sensing element array are smaller than the color image pixel array, the number of sensor elements is reduced under the condition that the color image pixels are not changed, the area of the sensor elements is relatively enlarged, and the signal-to-noise ratio of sampling is improved. Thereby improving the image acquisition quality of the color image sensor.
The color image sensing device can be widely applied to small and mobile devices such as mobile phones and the like, and the signal-to-noise ratio of color image signal acquisition is improved, so that the quality of color images is improved.
According to another embodiment of the present invention, there is provided a computing apparatus including the aforementioned image processing device. The computing device may be a smartphone, tablet, laptop, digital camera, digital video camera, and the like.
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data processing method according to various embodiments of the present application described in the above-mentioned "exemplary methods" section of this specification.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a data processing method according to various embodiments of the present application described in the "exemplary methods" section above of the present specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (15)
1. An image sensing device comprises one or two image sensors and one or two associated meta-color filter arrays, wherein the image sensors comprise sensor element arrays, the sensor elements in the sensor element arrays correspond to the color filter elements in the meta-color filter element arrays one by one,
one color filter element corresponds to one pixel, two pixels or three pixels, wherein one color filter element corresponds to a so-called single color filter element of one pixel, one color filter element corresponds to a so-called double color filter element of two pixels, one color filter element corresponds to a so-called triple color filter element of three pixels, the area of the double color filter element is twice that of the single color filter element, the area of the triple color filter element is three times that of the single color filter element, correspondingly, one sensor element corresponds to a so-called single sensor element of one pixel, one sensor element corresponds to a so-called double sensor element of two pixels, one sensor element corresponds to a so-called triple sensor element of three pixels, the area of the double sensor element is two times that of the single sensor element, and the area of the triple sensor element is three times that of the single sensor element,
wherein, at least one primary color filter element array comprises filter elements of at least two primary colors, and all the primary color filter element arrays form a full color filter element array with all the primary colors, wherein, the number of at least one of the double filter elements and the triple filter elements is not zero, correspondingly, the number of at least one of the double sensor elements and the triple sensor elements is not zero,
the color filter array comprises two or three of single color filter element, double color filter element and triple color filter element, the sensor array comprises two or three of corresponding single sensor element, double sensor element and triple sensor element,
the proportion among the three primary color filter elements in at least one element color filter element array accords with a preset proportion relation, and according to the preset proportion relation, single filter elements, double filter elements and triple filter elements of two or three colors are arranged randomly or meet a certain constraint condition to form a random mixed triple filter element array, wherein the number of at least one of the double filter elements and the triple filter elements is not zero.
2. The image sensing device of claim 1, comprising two image sensors and two arrays of metamer color filter elements,
all the primary colors are three primary colors or four primary colors, wherein the four primary colors are red, green, blue and colorless,
at least one of the two primary color filter arrays comprises at least two primary color filter elements, and all the two primary color filter elements form a full-color filter element array with all primary colors.
3. The image sensing device of claim 1, wherein the total primary colors are three primary colors RGB, and the predetermined ratio of the three primary colors is R: G: B ═ 1: 2: 1, wherein the red color filter element, the blue color filter element and the corresponding sensor element are a single color filter element and a single sensor element, and the green color filter element and the corresponding sensor element are a double color filter element and a double sensor element.
4. The image sensing device of claim 1, the all primary colors being one of white or colorless and red, green, and blue.
5. The image sensing device of claim 1, wherein only dual color filters are included in the array of color filters, and correspondingly only corresponding dual sensor cells are included in the array of sensor cells.
6. The image sensing device of claim 1, wherein only three color filters are included in the array of color filters, and correspondingly only three corresponding sensor cells are included in the array of sensor cells.
7. The image sensing device of claim 6, the tri-color filter and the tri-sensor element being L-shaped.
8. The image sensing device of claim 1, comprising two image sensors and associated two arrays of metamer color filter elements;
the first unitary color filter element array consists of a first primary color filter element and a second primary color filter element;
the second element color filter element array is composed of color filter elements of a third primary color, and the color filter elements of the first primary color, the second primary color and the third primary color are double color filter elements.
9. The image sensing device of claim 1, comprising two image sensors and associated two arrays of metamer color filter elements;
the first unitary color filter element array consists of a first primary color filter element and a second primary color filter element;
the second element color filter element array is composed of color filter elements of a third primary color and transparent color filter elements.
10. The image sensing device of claim 9, wherein the color filter of the first primary color, the color filter of the second primary color, the color filter of the third primary color, and the transparent color filter are all dual color filters.
11. A method of manufacturing the image sensing apparatus according to any one of claims 1 to 10, comprising:
determining the proportion between the three primary color filter elements;
randomly determining the number of single color filter elements, double color filter elements and triple color filter elements of the three colors and accordingly determining the number of corresponding single sensor elements, double sensor elements and triple sensor elements based on the proportion;
manufacturing single color filter elements, double color filter elements and triple color filter elements of the determined number of three colors, and manufacturing corresponding single sensor elements, double sensor elements and triple sensor elements;
and forming a random mixed three-color image sensor by manufacturing the determined number of single color filter elements, double color filter elements and three color filter elements of the three colors and corresponding single sensor elements, double sensor elements and three sensor elements.
12. An image processing apparatus comprising the image sensing apparatus according to any one of claims 1 to 10 and an image processor,
wherein the image processor obtains each corresponding sub-color image based on the signal sampled by each image sensor, reconstructs to obtain a final full-color image based on each corresponding sub-color image,
under the condition that an element color filter element array corresponding to one image sensor comprises a plurality of color filter elements of primary colors, the image processor obtains a monochromatic sub-color image corresponding to each primary color in the plurality of primary colors based on a signal sampled by the image sensor; under the condition that an element color filter element array corresponding to one image sensor comprises a single primary color filter element, an image processor obtains a single-color sub-color image corresponding to a single primary color based on a signal sampled by the image sensor; under the condition that a primary color filter element array corresponding to one image sensor comprises colorless filter elements, a corresponding gray level image is also obtained based on a signal sampled by the image sensor; and the image processor reconstructs the sub-color image and/or the sub-gray image with low total pixel amount obtained by the double color filter elements, the three color filter elements and the corresponding sensing elements by adopting demosaicing, noise reduction and super pixel methods to obtain the final color image and/or the gray image with high pixel amount based on the single sub-color image and/or the gray image corresponding to each image sensor.
13. A computing device comprising the image processing apparatus of claim 12.
14. The computing device of claim 13, the computing device being one of a smartphone, a tablet, a laptop, a digital camera, a digital video camera.
15. An image reconstruction method for reconstructing a color image using a signal obtained by the image sensing apparatus as claimed in any one of claims 1 to 10, comprising
Obtaining corresponding sub-color images based on signals sampled by the image sensors, reconstructing the sub-color images based on the corresponding sub-color images to obtain a final full-color image,
under the condition that a primary color filter element array corresponding to one image sensor comprises a plurality of primary color filter elements, obtaining a monochromatic sub-color image corresponding to each primary color in a plurality of primary colors based on a signal sampled by the image sensor;
under the condition that an element color filter element array corresponding to one image sensor comprises a single primary color filter element, obtaining a single-color sub-color image corresponding to a single primary color based on a signal sampled by the image sensor;
under the condition that a primary color filter element array corresponding to one image sensor comprises colorless filter elements, a corresponding gray level image is also obtained based on a signal sampled by the image sensor; and
based on the single sub-color image and/or gray level image corresponding to each image sensor, the sub-color image and/or sub-gray level image with low total pixel amount obtained by sampling by the double color filter elements, the three color filter elements and the corresponding sensing elements is reconstructed by adopting the methods of demosaicing, noise reduction and super pixel to obtain the final color image and/or gray level image with high pixel amount.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205091.8A CN111416967B (en) | 2020-03-20 | 2020-03-20 | Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010205091.8A CN111416967B (en) | 2020-03-20 | 2020-03-20 | Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111416967A CN111416967A (en) | 2020-07-14 |
| CN111416967B true CN111416967B (en) | 2021-10-08 |
Family
ID=71493304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010205091.8A Active CN111416967B (en) | 2020-03-20 | 2020-03-20 | Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111416967B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578078A (en) * | 2015-12-18 | 2016-05-11 | 广东欧珀移动通信有限公司 | Image sensor, imaging device, mobile terminal and imaging method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107529046B (en) * | 2017-02-23 | 2024-03-08 | 思特威(深圳)电子科技有限公司 | Color filter array and image sensor |
-
2020
- 2020-03-20 CN CN202010205091.8A patent/CN111416967B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105578078A (en) * | 2015-12-18 | 2016-05-11 | 广东欧珀移动通信有限公司 | Image sensor, imaging device, mobile terminal and imaging method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111416967A (en) | 2020-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI504276B (en) | Image sensor for capturing a color image | |
| CN111614886B (en) | Image sensor and electronic device | |
| CN112529775A (en) | Image processing method and device | |
| CN102577395B (en) | Solid-state image pickup element and image pickup apparatus | |
| EP3054675B1 (en) | Imaging systems with clear filter pixels | |
| CN110463197B (en) | Enhancing spatial resolution in stereoscopic camera imaging systems | |
| CN109417613B (en) | Image sensor method and apparatus having multiple successive infrared filtering units | |
| JP2009511960A (en) | System and method for high performance color filter mosaic array | |
| CN103035662B (en) | Image device, electronic equipment and manufacture method | |
| EP3902242B1 (en) | Image sensor and signal processing method | |
| EP2685725B1 (en) | Imaging device and imaging program | |
| US20210297607A1 (en) | Sparse infrared pixel design for image sensors | |
| US9219894B2 (en) | Color imaging element and imaging device | |
| CN114693580B (en) | Image processing method and related device | |
| CN104410786B (en) | The control method of image processing equipment and image processing equipment | |
| CN110225232A (en) | Image processing apparatus | |
| CN101288296A (en) | Reconstructing a full color image from an image encoded by a bayer pattern | |
| US9185375B2 (en) | Color imaging element and imaging device | |
| US9143747B2 (en) | Color imaging element and imaging device | |
| US20230007191A1 (en) | Image sensor, imaging apparatus, electronic device, image processing system, and signal processing method | |
| CN101282486A (en) | Image processing device | |
| CN106791734A (en) | The method of device, electronic installation and IMAQ for IMAQ | |
| KR20150063010A (en) | Method and device for estimating disparity associated with views of a scene acquired with a plenoptic camera | |
| CN208820948U (en) | Image sensing apparatus | |
| CN111416967B (en) | Image sensing apparatus, method of manufacturing the same, image reconstruction method, and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |