CN114097218A - Imaging device, imaging method and camera - Google Patents
Imaging device, imaging method and camera Download PDFInfo
- Publication number
- CN114097218A CN114097218A CN202080031334.9A CN202080031334A CN114097218A CN 114097218 A CN114097218 A CN 114097218A CN 202080031334 A CN202080031334 A CN 202080031334A CN 114097218 A CN114097218 A CN 114097218A
- Authority
- CN
- China
- Prior art keywords
- green channel
- channel
- range interval
- green
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 71
- 230000035945 sensitivity Effects 0.000 claims abstract description 69
- 230000007935 neutral effect Effects 0.000 claims abstract description 21
- 238000013507 mapping Methods 0.000 claims description 45
- 238000012937 correction Methods 0.000 claims description 14
- 238000012886 linear function Methods 0.000 claims description 11
- 238000000034 method Methods 0.000 claims description 8
- 230000002238 attenuated effect Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 239000002184 metal Substances 0.000 description 8
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Electromagnetism (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
An imaging apparatus, an imaging method, and a camera, including: an image sensor array of four color channels including a red channel (R), a blue channel (B), a first green channel (Gr), and a second green channel (Gb); wherein a neutral gray filter (5) is arranged on a first green channel (Gr) such that the sensitivity of the first green channel (Gr) is lower than the sensitivity of the second green channel (Gb); alternatively, the gain of the analog-to-digital converter ADC on the first green channel (Gr) is smaller than the gain of the ADC on the second green channel (Gb), such that the sensitivity of the first green channel (Gr) is lower than the sensitivity of the second green channel (Gb). By providing the imaging device, the imaging method and the camera, imaging with a high dynamic range can be obtained through a simple circuit and at a low cost.
Description
The present invention relates to the field of imaging technologies, and in particular, to an imaging apparatus, an imaging method, and a camera.
The invention discloses a novel high-dynamic range (HDR) CIS (COMS IMAGE SENSOR) imaging device and an imaging method, which are applied to a mobile phone camera, a digital camera or a digital video camera.
In the existing HDR CIS imaging device, for example, the IMX586 of SONY, which is a qbc (quad bayer coding) CIS, the principle is to synthesize four different exposure pixels of quad bayer to obtain a pixel with high dynamic range. However, the above-described imaging device has a disadvantage in that only one of four pixels can be used for a pixel for recovering a highlight portion, which reduces an effective utilization area of a sensor, resulting in low image quality of an entire output. In addition, the tone mapping (tone mapping) design of the original pixels (raw) results in complex circuitry of the sensor, and significant cost and power consumption.
In a non-HDR CIS imaging device, the HDR purpose is usually achieved by fusing a plurality of differently exposed photos, and the method has the unified disadvantage of being sensitive to motion blur and easily causing defects in a scene with a moving object. Moreover, multiple exposures are easy to be fused into flaws due to hand shaking at night. In addition, when the exposure time cannot be continuously reduced on a sunny day, the highlight information cannot be recovered by a plurality of exposures. Finally, the fusion occupies a large amount of computing resources, and no special chip can be used for HDR video.
Therefore, it is necessary to provide an imaging apparatus, an imaging method, and a camera to solve the above problems.
Disclosure of Invention
The present invention in its first aspect provides an image forming apparatus comprising: an image sensor array of four color channels, the four color channels including a red channel, a blue channel, a first green channel, and a second green channel; a neutral gray filter is arranged on the first green channel, so that the sensitivity of the first green channel is lower than that of the second green channel; or the gain of the analog-to-digital converter (ADC) on the first green channel is smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than that of the second green channel.
In a second aspect, an embodiment of the present invention further provides an imaging method, where the first green channel is attenuated; restoring highlight information of the pixel by using the sensitivity of the attenuated first green channel; wherein a neutral gray filter is disposed on the first green channel such that a sensitivity of the first green channel is lower than a sensitivity of the second green channel; or, the gain of the ADC on the first green channel is reduced to be smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than the sensitivity of the second green channel.
In a third aspect, an embodiment of the present invention further provides a camera, including: the imaging apparatus according to the first aspect, or the imaging method according to the second aspect.
By providing the imaging device, the imaging method and the camera, the invention can obtain imaging with high dynamic range through a simple circuit and with low cost.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a first bayer array backside-illuminated CIS according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a second bayer array backside-illuminated CIS according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a luminance mapping curve of a first green channel in a second bayer array back-illuminated CIS structure according to an embodiment of the present invention;
fig. 4 is a comparison graph of sensor effects generated by a first bayer array back-illuminated CIS structure and a second bayer array back-illuminated CIS structure according to an embodiment of the present invention;
FIG. 5 is a first partial comparison graph of the sensor effect comparison graph of FIG. 4;
fig. 6 is a second partial comparison of the sensor effect comparison shown in fig. 4.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic structural diagram of a first bayer array backside-illuminated CIS according to an embodiment of the present invention.
Specifically, the first bayer array backside-illuminated CIS structure mainly includes the following main components: microlenses, color filters, metal light shields, metal interconnects, photodiodes, and the like.
As shown in fig. 1, which shows a planar structure of a color filter, and shows a schematic diagram of a bayer array backside-illuminated CIS structure, which includes a microlens 1, a color filter 2, a metal light shield 3, a metal interconnection 4, and a photodiode (not shown) in a stacked arrangement, wherein the Photodiode (PD) is used to capture light, and exemplarily, the photodiode may be selected as a PIN diode or a PN junction device. The color filter 2 is used to separate red, green, and blue (RGB) components of the reflected light. Finally, the microlens 1 collects light from the inactive portion of the CIS and focuses it to the photodiode.
Illustratively, the microlenses may be selected to be spherical surfaces or mesh lenses.
The structure of the color filter 2 is exemplarily shown in fig. 1, which includes four channels R, Gr, Gb, and B, where R is a red channel, Gr is a first green channel, Gb is a second green channel, and B is a blue channel. Wherein the sensitivity of the channels Gr, Gb is the same.
The present application also provides an image forming apparatus including: an image sensor array of four color channels, the four color channels including a red channel, a blue channel, a first green channel, and a second green channel; a neutral gray filter is arranged on the first green channel, so that the sensitivity of the first green channel is lower than that of the second green channel; or the gain of the analog-to-digital converter (ADC) on the first green channel is smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than that of the second green channel.
Correspondingly, an imaging method is also provided, and the first green channel is attenuated; restoring highlight information of the pixel by using the sensitivity of the attenuated first green channel; wherein a neutral gray filter is disposed on the first green channel such that a sensitivity of the first green channel is lower than a sensitivity of the second green channel; or, the gain of the ADC on the first green channel is reduced to be smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than the sensitivity of the second green channel.
Fig. 2 is a schematic structural diagram of a second bayer array backside-illuminated CIS according to an embodiment of the present invention.
Specifically, the second bayer array backside-illuminated CIS structure mainly includes the following main components: microlenses, neutral gray filters, color filters, metal masks, metal interconnects, photodiodes, and the like.
As shown in fig. 2, which shows a planar structure of a color filter, and shows a schematic diagram of a second bayer array backside-illuminated CIS structure, including a microlens 1, a neutral gray filter 5, a color filter 2, a metal light shield 3, a metal interconnection 4, and a photodiode (not shown) in a stacked arrangement, wherein the Photodiode (PD) is used for capturing light, and the photodiode may be selected as a PIN diode or a PN junction device, for example. The color filter is used to separate red, green, and blue (RGB) components of the reflected light. Finally, the microlens 1 collects light from the inactive portion of the CIS and focuses it to the photodiode.
Illustratively, the microlenses may be selected to be spherical surfaces or mesh lenses.
The structure of the color filter 2 is exemplarily shown in fig. 2, which includes four channels R, Gr, Gb, and B, where R is a red channel, Gr is a first green channel, Gb is a second green channel, and B is a blue channel.
Illustratively, an image sensor array of four color channels including a red channel, a blue channel, a first green channel, and a second green channel; a neutral gray filter is arranged on the first green channel, so that the sensitivity of the first green channel is lower than that of the second green channel; or the gain of the analog-to-digital converter (ADC) on the first green channel is smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than that of the second green channel.
As shown in fig. 2, in the second bayer array backside-illuminated CIS structure, a neutral gray filter 5(ND filter) is added in front of one of the G channels, for example, the Gb channel, so that the sensitivity of the Gb channel is lower than that of the Gr channel. Although fig. 2 exemplarily shows that the neutral gray filter 5 is disposed in front of the Gb channel, it can be understood by those skilled in the art that the neutral gray filter 5 can also be disposed in front of the Gr channel in other embodiments, and similarly, the following description will be given by setting the gain of the neutral gray filter 5 or the analog-to-digital converter ADC on the Gb channel to be smaller than that on the Gr channel, so that the sensitivity of the Gb channel is lower than that of the Gr channel, as an example, but replacing it with setting the gain of the neutral gray filter 5 or the analog-to-digital converter ADC on the Gr channel to be smaller than that on the Gb channel, so that the sensitivity of the Gr channel is lower than that of the Gb channel can also achieve the same effect and achieve the same technical effect.
Illustratively, the sensitivity of the first green channel may also be made lower than the sensitivity of the second green channel by the gain of the analog-to-digital converter ADC on the first green channel being smaller than the gain of the ADC on the second green channel.
The light receives an electrical signal after being subjected to photoelectric conversion by the photodiode, and the electrical signal after the photoelectric conversion is usually an analog signal, so for convenience of subsequent processing, in the bayer array back-illuminated CIS structure, a digital-to-analog converter ADC is often arranged on each color channel to convert the analog electrical signal into a digital electrical signal, and in the process of performing analog-to-digital conversion, the gain of the analog-to-digital converter ADC on the first green channel may be set to be smaller than the ADC gain on the second green channel, so that even if the same light is respectively incident on the first green channel and the second green channel, the sensitivity of the first green channel may be lower than the sensitivity of the second green channel because the gain of the ADC on the first green channel is smaller than the gain of the ADC on the second green channel.
In this case, the sensitivity of Gr is high, and as the flux of incident light becomes larger from a small one, the Gr channel reaches overexposure before the Gb channel. At a certain moment, when the Gr channel is over-exposed, the Gb channel is not over-exposed, so that the high-light information of a part can be recovered by utilizing the sensitivity of the Gr channel or combining the sensitivities of the Gr channel and the Gb channel.
For example, when the camera shoots a scene with a high dynamic range such as a backlight on a sunny day, a cloud on a sunset day or an LED lamp at night, a principle that the sensitivity of one G channel is relatively low can be utilized to recover part of highlight information.
Therefore, according to the technical scheme of the application, only by arranging a neutral gray filter or reducing the gain of an ADC, the structural change is small or the structure of a new imaging device is not required to be added at all, and imaging with a high dynamic range can be obtained only by changing the existing device. High dynamic range imaging is thus obtained with relatively simple circuitry and at relatively low cost.
Illustratively, the sensitivity of the red channel is the same as the sensitivity of the blue channel. As shown in fig. 2, no neutral gray filter is disposed above the R channel and the B channel, and no ADC gain of the R channel or the B channel is particularly set, so that the sensitivity of the R channel and the sensitivity of the B channel are the same, and it is ensured that the optical signals of the red channel and the green channel are sufficiently restored, and the authenticity of the imaged image is further ensured on the basis of achieving the above technical effects.
Illustratively, the sensitivity of the first green channel is less than or equal to 1/2 of the sensitivity of the second green channel.
Illustratively, the imaging device further comprises a first processor for recovering highlight information of the imaging device by using the sensitivity of the first green channel when the overexposure of the second green channel is less than or equal to 1 EV.
Illustratively, when overexposure is performed on the Gr channel but not on the 1EV overexposure, the Gb channel is not overexposed, and highlight information of the location area can be recovered using the Gb channel or combining signals of the Gr channel and the Gb channel.
Here, 1EV means that, for example, when the sensitivity is ISO 100, the aperture ratio is F1, and the exposure time is 1 second, the exposure amount is defined as 0, the exposure amount is decreased by one step (the shutter time is decreased by half or the aperture is increased by one step), and the EV value is increased by 1.
When the sensitivity of the Gb channel is equal to 1/2 of the Gr channel sensitivity, when the Gr channel is overexposed but there is no overexposure 1EV, the Gb channel is not yet overexposed, at this time, the Gb channel can relatively truly reflect the object to be imaged, and since there is already 1EV overexposed in the Gr channel, part of highlight information is lost, at this time, part of highlight information of the object to be imaged can be recovered using the Gb channel.
When the sensitivity of the Gb channel is less than 1/2 of the Gr channel sensitivity, even if the Gr channel overexposure exceeds 1EV, the Gb channel may still be in a non-overexposed state, at this time, the Gb channel can still relatively truly reflect the object to be imaged, and because the Gr channel has been overexposed, where part or even all highlight information is lost, at this time, all or part of the highlight information of the object to be imaged can be recovered using the Gb channel.
Therefore, on the basis of realizing the technical effects of the Gb channels, the authenticity of an imaging image is further ensured, and the imaging quality of an object to be imaged can be ensured in a larger overexposure range.
Illustratively, the image sensor is in a Raw-Bayer format.
In the Raw-Bayer format, by setting two green channels, i.e., the first green channel and the second green channels Gr and Gb, it is possible to realize that the sensitivity of one of the green channels is smaller than that of the other green channel.
In other embodiments, the image sensor may also adopt other formats, which is not limited in this application.
Exemplarily, the method further comprises the following steps: a second processor that pre-processes signals of a red channel, a blue channel, a first green channel, and a second green channel.
In order to obtain a better imaging effect, such a bayer CIS with a neutral gray filter (ND filter)5 added thereto as shown in fig. 2 needs to change a response in an ISP Image Signal Processor (Image Signal Processor) link, for example, to preprocess signals of four channels.
Illustratively, the preprocessing includes Demosaic processing. Or, the restoring highlight information of the pixel by using the attenuated sensitivity of the first green channel includes: and (5) removing treatment.
The Demosaic processing may select a bilinear interpolation method or an adaptive interpolation method.
Illustratively, the Demosaic processing includes mapping the first green channel signal and the second green channel signal to a first green correction signal and a second green correction signal, respectively.
Illustratively, prior to the mapping, the signals of the first and second green channels are normalized.
As shown in fig. 3, signals of the first green channel Gb and the second green channel Gr are mapped to Gnew, respectively.
Specifically, it is necessary to design a luminance mapping curve of a green channel so that Gr and Gb channels of high and low light portions can be matched, and fig. 3 shows a luminance mapping curve of a green channel, and the abscissa and ordinate are normalized ambient luminance and green channel response, respectively, and Gr and Gb are mapped to a Gnew curve, respectively.
Illustratively, the mapping functions adopted when the signal values of the first green channel are in a first range interval and a second range interval are different, the first range interval and the second range interval are different, and the values in the first range interval are smaller than the values in the second range interval.
Specifically, different mapping functions are adopted for signals of Gb channels in different range intervals, for example, the signal values of Gb are divided into two range intervals, and in other embodiments, the Gb signals may also be divided into other number of range intervals, which is not limited in this application.
Illustratively, the mapping function of the signal of the first green channel is a direct proportional function when the signal value of the first green channel is in the first range interval; when the signal value of the first green channel is in the second range interval, the mapping function of the signal of the first green channel is a linear function without exceeding the origin of the coordinate system.
Specifically, in a first range section with a small signal value, the adopted mapping function makes a direct proportional relationship between signals of the Gb channel and the processed signals Gb ', when the signals of the Gb channel increase, in a second range section, the processed signals are formed with a relatively small slope, so that the processed signals Gb' reach gain saturation or overexposure values relatively slowly, and meanwhile, in order to ensure continuity of the signals of the two range sections, the adopted mapping function in the second range section is a linear function without passing through the origin of the coordinate system.
Illustratively, when the neutral gray filter 5 selects ND2, i.e., the Gb channel sensitivity is 1/2 of the Gr channel sensitivity, the mapping formula for Gb is as follows:
specifically, the first range section of Gb is (0, 0.375), the second range section is (0.375, 1), the mapping function of the signals of the Gb channel is a proportional function 4/3Gb in the first range section, and the mapping function of the signals of the Gb channel is a linear function 4/5(Gb-0.375) +0.5 without exceeding the origin of the coordinate system in the second range section.
Where Gb' is the first green correction signal.
Illustratively, the mapping functions adopted when the signal values of the second green channel are in a third range interval, a fourth range interval and a fifth range interval are different, the third range interval, the fourth range interval and the fifth range interval are different, the values in the third range interval are smaller than the values in the fourth range interval, and the values in the fourth range interval are smaller than the values in the fifth range interval.
Illustratively, the mapping function of the signal of the second green channel is a direct proportional function when the signal value of the second green channel is in the third range interval; when the signal value of the second green channel is in the fourth range interval, the mapping function of the signal of the second green channel is a linear function without exceeding the origin of the coordinate system; when the signal value of the second green channel is in the fifth range interval, the signal of the second green channel is the same as the second green correction signal.
Specifically, in a third range section with a smaller signal value, the adopted mapping function makes a direct proportional relationship between the signal of the Gr channel and the processed signal Gr ', when the signal of the Gr channel increases, in a fourth range section, the adopted mapping function is a linear function that does not exceed the origin of the coordinate system, when the signal of the Gr channel continues to increase, in a fifth range section, the adopted mapping function is a linear function that does not exceed the origin of the coordinate system, and Gr ' is the same as the first green correction signal, which is selected because, at this time, the Gr channel has reached an overexposed state, and the signal value read from the Gr channel itself has failed to reflect the information of the object to be imaged, and therefore, the value thereof is directly selected as Gb ' that is not in the overexposed state. Since Gb' more accurately reflects the state of the image to be imaged, this approach allows some or even all of the highlight information to be recovered in the case of overexposure of one green channel.
Illustratively, when the neutral gray filter 5 selects ND2, i.e., the Gb channel sensitivity is 1/2 of the Gr channel sensitivity, the mapping formula for Gr is as follows:
specifically, the third range interval of Gr is (0, 0.75), the fourth range interval is (0.75, 1), and the fifth range interval is 1. in the first range interval, the mapping function of the signals of the Gb channels is a proportional function 4/3Gb, in the second range interval, the mapping function of the signals of the Gb channels is a linear function 4/5(Gb-0.375) +0.5 without passing through the origin of the coordinate system, and in the fifth range interval, Gr' directly follows the value of the first green-color-corrected signal.
Where Gr' is the second green color correction signal.
Through the mapping process, the mapping relationship shown in fig. 3 is formed, and it should be noted that the graph shown in fig. 3 only describes the mapping rule qualitatively, and does not represent a quantitative value.
Illustratively, the second processor, in pre-processing the red and blue channels, comprises: and correspondingly reducing according to the reduction multiple of the first green correction signal compared with the first green channel signal.
Specifically, after the green channel is processed by the mapping, the red channel and the blue channel need to be processed correspondingly.
Illustratively, the R, B channels are modified. After the G channel is processed, the R and B channels may be correspondingly corrected by solving the reduction factor of the Gr channel at each position of the whole graph, and Gr _ mesh is calculated first as Gr'/Gr.
R'＝R.*Gr_mesh
B'＝B.*Gr_mesh
Wherein, R 'is the corrected red channel signal, and B' is the corrected green channel signal.
Illustratively, in one embodiment, the remaining ISP modules are identical to the conventional ISP modules without any difference, except that demosaic needs to do the preprocessing.
The imaging device and the imaging method can be exemplarily applied to any recording equipment such as a camera, a video camera and the like capable of recording images, videos and the like, so that when the equipment such as the camera and the like shoots a scene with a high dynamic range, such as a backlight in sunny days, a cloud in sunset days or an LED lamp at night, a principle that the sensitivity of a G channel is relatively low can be utilized to recover partial highlight information.
Fig. 4 is a comparison graph of sensor effects generated by a first bayer array back-illuminated CIS structure and a second bayer array back-illuminated CIS structure according to an embodiment of the present invention; FIG. 5 is a first partial comparison graph of the sensor effect comparison graph of FIG. 4; fig. 6 is a second partial comparison of the sensor effect comparison shown in fig. 4.
According to the imaging device and the imaging method provided by the invention, the same object is imaged under the same environment, the left picture and the right picture shown in fig. 4 are respectively obtained, and the details of the two pictures in fig. 4 are respectively shown in fig. 5 and fig. 6.
The sensor effect map on the right is clearly improved from that shown in fig. 4, and a clear recovery is achieved for the position of overexposure in the image, as can be seen in conjunction with fig. 5 and 6, wherein more details are shown in the picture on the right, ensuring the authenticity and accuracy of the picture.
According to the technical scheme, only one neutral gray filter is arranged or the gain of one ADC is reduced, the structure is changed slightly or changed differently at all, and imaging in a high dynamic range can be obtained only by changing the existing device. High dynamic range imaging is obtained with relatively simple circuitry and at relatively low cost.
Technical terms used in the embodiments of the present invention are only used for illustrating specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, the use of "including" and/or "comprising" in the specification is intended to specify the presence of stated features, integers, steps, operations, elements, and/or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and/or components.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below, if any, are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Various modifications and alterations to this invention will become apparent to those skilled in the art without departing from the scope and spirit of this invention. The embodiments described herein are further intended to explain the principles of the invention and its practical application and to enable others skilled in the art to understand the invention.
The flow chart described in the present invention is only an example, and various modifications can be made to the diagram or the steps in the present invention without departing from the spirit of the present invention. For instance, the steps may be performed in a differing order, or steps may be added, deleted or modified. It will be understood by those skilled in the art that all or a portion of the above-described embodiments may be practiced and equivalents thereof may be resorted to as falling within the scope of the invention as claimed.
Claims (29)
- An image forming apparatus, comprising: an image sensor array of four color channels, the four color channels including a red channel, a blue channel, a first green channel, and a second green channel;a neutral gray filter is arranged on the first green channel, so that the sensitivity of the first green channel is lower than that of the second green channel; alternatively, the first and second electrodes may be,the gain of the analog-to-digital converter (ADC) on the first green channel is less than the gain of the ADC on the second green channel, such that the sensitivity of the first green channel is lower than the sensitivity of the second green channel.
- The imaging apparatus of claim 1, wherein a sensitivity of the first green channel is less than or equal to 1/2 of a sensitivity of the second green channel.
- The imaging apparatus of claim 1, wherein a sensitivity of the red channel is the same as a sensitivity of the blue channel.
- The imaging apparatus according to claim 1 or 2, further comprising a first processor configured to recover highlight information of the imaging apparatus using the sensitivity of the first green channel when the overexposure of the second green channel is 1EV or less.
- The imaging apparatus of any of claims 1 to 4, wherein the image sensor is in a Raw-Bayer format.
- The imaging apparatus of claim 1, further comprising:a second processor that pre-processes signals of a red channel, a blue channel, a first green channel, and a second green channel.
- The imaging apparatus of claim 6, wherein the pre-processing comprises Demosaic processing.
- The imaging apparatus of claim 6 or 7, wherein the Demosaic processing comprises mapping the first and second green channel signals to first and second green correction signals, respectively.
- The imaging apparatus according to claim 8, wherein the mapping function adopted when the signal value of the first green channel is in a first range interval and a second range interval is different, the first range interval and the second range interval are different, and the value in the first range interval is smaller than the value in the second range interval.
- The imaging apparatus of claim 9, wherein the mapping function of the signal of the first green channel is a direct proportional function when the signal value of the first green channel is in the first range interval;when the signal value of the first green channel is in the second range interval, the mapping function of the signal of the first green channel is a linear function without exceeding the origin of the coordinate system.
- The imaging apparatus according to claim 8, wherein the mapping functions applied to the signal values of the second green channel are different between a third range interval, a fourth range interval and a fifth range interval, the third range interval, the fourth range interval and the fifth range interval are different, and the values in the third range interval are smaller than the values in the fourth range interval, and the values in the fourth range interval are smaller than the values in the fifth range interval.
- The imaging apparatus of claim 11, wherein the mapping function of the signal of the second green channel is a direct proportional function when the signal value of the second green channel is in the third range interval;when the signal value of the second green channel is in the fourth range interval, the mapping function of the signal of the second green channel is a linear function without exceeding the origin of the coordinate system;when the signal value of the second green channel is in the fifth range interval, the signal of the second green channel is the same as the second green correction signal.
- The imaging apparatus of claim 6, wherein the signals of the first and second green channels are normalized prior to the mapping.
- The imaging apparatus of claim 6, wherein the second processor, in pre-processing the red and blue channels, comprises:and correspondingly reducing according to the reduction multiple of the first green correction signal compared with the first green channel signal.
- An imaging method, characterized by attenuating a first green channel;restoring highlight information of the pixel by using the sensitivity of the attenuated first green channel;wherein a neutral gray filter is disposed on the first green channel such that a sensitivity of the first green channel is lower than a sensitivity of the second green channel; alternatively, the first and second electrodes may be,and reducing the gain of the analog-to-digital converter (ADC) on the first green channel to be smaller than the gain of the ADC on the second green channel, so that the sensitivity of the first green channel is lower than that of the second green channel.
- The imaging method of claim 15, wherein a sensitivity of the first green channel is less than or equal to 1/2 of a sensitivity of the second green channel.
- The imaging method of claim 15, further comprising a red channel and a blue channel, wherein a sensitivity of the red channel is the same as a sensitivity of the blue channel.
- The imaging method according to claim 15, wherein highlight information of the imaging device is restored using the sensitivity of the first green channel when overexposure of the second green channel is 1EV or less.
- The imaging method of claim 15, applied to Raw-Bayer-format pixels.
- The imaging method of claim 15, further comprising:signals of a red channel, a blue channel, a first green channel, and a second green channel are preprocessed.
- The imaging method of claim 15, wherein said recovering highlight information for a pixel using the attenuated sensitivity of the first green channel comprises: and (5) removing treatment.
- The imaging method of claim 20, further comprising mapping the first green channel signal and the second green channel signal to a first green correction signal and a second green correction signal, respectively.
- The imaging method according to claim 22, wherein the mapping function used for the signal values of the first green channel is different between a first range interval and a second range interval, wherein the first range interval and the second range interval are different, and wherein the values in the first range interval are smaller than the values in the second range interval.
- The imaging method of claim 21, wherein the mapping function of the signal of the first green channel is a direct proportional function when the signal value of the first green channel is in the first range interval;when the signal value of the first green channel is in the second range interval, the mapping function of the signal of the first green channel is a linear function without exceeding the origin of the coordinate system.
- The imaging method according to claim 22, wherein the mapping functions used for the signal values of the second green channel are different between a third range interval, a fourth range interval and a fifth range interval, wherein the third range interval, the fourth range interval and the fifth range interval are different, and wherein the values in the third range interval are smaller than the values in the fourth range interval, and wherein the values in the fourth range interval are smaller than the values in the fifth range interval.
- The imaging method of claim 25, wherein the mapping function of the signal of the second green channel is a direct proportional function when the signal value of the second green channel is in the third range interval;when the signal value of the second green channel is in the fourth range interval, the mapping function of the signal of the second green channel is a linear function without exceeding the origin of the coordinate system;when the signal value of the second green channel is in the fifth range interval, the signal of the second green channel is the same as the second green correction signal.
- The imaging method according to claim 18, wherein the signals of the first green channel and the second green channel are normalized before the mapping.
- The imaging method of claim 18, wherein the second processor, in pre-processing the red and blue channels, comprises:and correspondingly reducing according to the reduction multiple of the first green correction signal compared with the first green channel signal.
- A camera, comprising:the imaging apparatus of any one of claims 1 to 14, or the imaging method of any one of claims 15 to 28.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/088466 WO2021217640A1 (en) | 2020-04-30 | 2020-04-30 | Imaging device, imaging method and camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114097218A true CN114097218A (en) | 2022-02-25 |
Family
ID=78373175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080031334.9A Pending CN114097218A (en) | 2020-04-30 | 2020-04-30 | Imaging device, imaging method and camera |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114097218A (en) |
| WO (1) | WO2021217640A1 (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000041189A (en) * | 1998-07-22 | 2000-02-08 | Sony Corp | Image pickup device |
| CN108364966A (en) * | 2018-02-06 | 2018-08-03 | 德淮半导体有限公司 | Imaging sensor and forming method thereof |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101909157B (en) * | 2009-06-05 | 2012-09-05 | 比亚迪股份有限公司 | Method for acquiring highly dynamic images from image sensor, and image sensor |
| JP2015133633A (en) * | 2014-01-14 | 2015-07-23 | 株式会社東芝 | Solid state imaging device |
| CN108400143A (en) * | 2018-02-28 | 2018-08-14 | 德淮半导体有限公司 | Imaging sensor and forming method thereof |
-
2020
- 2020-04-30 WO PCT/CN2020/088466 patent/WO2021217640A1/en active Application Filing
- 2020-04-30 CN CN202080031334.9A patent/CN114097218A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000041189A (en) * | 1998-07-22 | 2000-02-08 | Sony Corp | Image pickup device |
| CN108364966A (en) * | 2018-02-06 | 2018-08-03 | 德淮半导体有限公司 | Imaging sensor and forming method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021217640A1 (en) | 2021-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11706535B2 (en) | Digital cameras with direct luminance and chrominance detection | |
| CN212785522U (en) | Image sensor and electronic device | |
| US7940311B2 (en) | Multi-exposure pattern for enhancing dynamic range of images | |
| CN112261391B (en) | Image processing method, camera assembly and mobile terminal | |
| CN102365861B (en) | Exposing pixel groups in producing digital images | |
| US8179445B2 (en) | Providing improved high resolution image | |
| US8169518B2 (en) | Image pickup apparatus and signal processing method | |
| US9270875B2 (en) | Dual image capture processing | |
| EP3439282A1 (en) | Image pickup device, image processing device, and electronic apparatus | |
| JP2009049525A (en) | Imaging apparatus and method for processing signal | |
| Kumar et al. | New digital camera sensor architecture for low light imaging | |
| WO2022073364A1 (en) | Image obtaining method and apparatus, terminal, and computer readable storage medium | |
| CN114097218A (en) | Imaging device, imaging method and camera | |
| CN114584725A (en) | Image sensor and imaging device | |
| JP2009049524A (en) | Imaging apparatus and method for processing signal | |
| JP5550333B2 (en) | Imaging apparatus, development method, and program | |
| JP7447947B2 (en) | Electronics | |
| KR102649298B1 (en) | Signal processing apparatus and signal processing method | |
| KR20230055626A (en) | Imaging Device | |
| JP2020078044A (en) | Signal processing apparatus and signal processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |