CN106507018B - Control method, control device and electronic device - Google Patents

Control method, control device and electronic device Download PDF

Info

Publication number
CN106507018B
CN106507018B CN201611079187.4A CN201611079187A CN106507018B CN 106507018 B CN106507018 B CN 106507018B CN 201611079187 A CN201611079187 A CN 201611079187A CN 106507018 B CN106507018 B CN 106507018B
Authority
CN
China
Prior art keywords
pixel
image
unit
electronic device
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611079187.4A
Other languages
Chinese (zh)
Other versions
CN106507018A (en
Inventor
韦怡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201611079187.4A priority Critical patent/CN106507018B/en
Publication of CN106507018A publication Critical patent/CN106507018A/en
Application granted granted Critical
Publication of CN106507018B publication Critical patent/CN106507018B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

The present invention discloses a kind of control method of electronic device.Firstly, control imaging sensor output merges image.Secondly, judging whether ambient brightness is less than or equal to preset threshold according to the system information of electronic device.Then, when ambient brightness is less than or equal to preset threshold, image will be merged and be converted to merging rgb image.Then, when ambient brightness is greater than preset threshold, patch image is simultaneously converted to imitative original image by control imaging sensor output patch image.Finally, imitative original image is converted to imitative former rgb image.In addition, invention additionally discloses a kind of control device and electronic devices.Control method, control device and electronic device of the invention judges ambient brightness using the system information of electronic device, it controls imaging sensor and exports suitable image, avoid the fixed outputting high quality image bring extensive work of imaging sensor, to reduce electronic device functions duration, working efficiency is improved, user satisfaction is improved.

Description

Control method, control device and electronic device
Technical field
The present invention relates to imaging technique, in particular to a kind of control method, control device and electronic device.
Background technique
A kind of existing imaging sensor includes sensitive pixel elements array and is arranged on sensitive pixel elements array Filter unit array, each filter unit array cover a corresponding sensitive pixel elements, each sensitive pixel elements packet Include multiple photosensitive pixels.When work, it can control image sensor exposure output and merge image, image can be passed through by merging image Processing method, which is converted to, to be merged rgb image and preserves.Merging image includes merging pixel array, same photosensitive pixel list Multiple photosensitive pixels of member merge output and are used as corresponding merging pixel.In this way, the signal-to-noise ratio for merging image can be improved, so And the resolution for merging image reduces.It is of course also possible to the patch image that image sensor exposure exports high pixel is controlled, color Block image includes original pixel array, the corresponding original pixels of each photosensitive pixel.However, due to same filter unit pair The multiple original pixels colors answered are identical, can not equally improve the resolution of patch image.Therefore, it is necessary to pass through interpolation calculation High pixel patch image is converted to the imitative original image of high pixel by mode, and imitative original image may include in the imitative of Bayer array arrangement Original pixel.Imitative original image can be converted to imitative former rgb image and be preserved by image processing method.However, interpolation calculation Expend resource and time-consuming, and and not all scene be all suitable for or need to export imitative original rgb image.
Summary of the invention
The present invention is directed at least solve one of the technical problems existing in the prior art.For this purpose, the present invention needs to provide one Kind control method, control device and electronic device.
A kind of control method, for controlling electronic device, the electronic device includes imaging device, the imaging device packet Imaging sensor is included, described image sensor includes sensitive pixel elements array and is arranged on the sensitive pixel elements array Filter unit array, each filter unit array covers the corresponding sensitive pixel elements, each described Sensitive pixel elements include multiple photosensitive pixels, the control method the following steps are included:
It controls described image sensor output and merges image, the merging image includes merging pixel array, same described Multiple photosensitive pixels of sensitive pixel elements merge output and are used as a merging pixel；
Judge whether the ambient brightness is less than or equal to preset threshold, the system according to the system information of the electronic device Information of uniting includes system time；
When the ambient brightness is less than or equal to preset threshold, the merging image is converted to merging rgb image.
When the ambient brightness is greater than preset threshold, control described image sensor exports patch image, the color lump Image includes the image pixel elements of predetermined array arrangement, and described image pixel unit includes multiple original pixels, each described Photosensitive pixel corresponds to the original pixels；
The patch image is converted to imitative original image, the imitative original image includes the imitative original pixel of array arrangement, described Imitative original pixel includes current pixel, and the original pixels include associated pixel corresponding with the current pixel, it is described will be described Patch image be converted to the step of imitative original image the following steps are included:
Whether the color for judging the current pixel is identical as the color of the associated pixel；
When the color of the current pixel is identical as the color of the associated pixel, by the pixel value of the associated pixel Pixel value as the current pixel；With
In the color difference of the color and the associated pixel of the current pixel, according to the pixel of associated pixel unit Value calculates the pixel value of the current pixel by the first interpolation algorithm, and described image pixel unit includes the associated pixel list The color of member, the associated pixel unit is identical as the current pixel and adjacent with the associated pixel；With
The imitative original image is converted to imitative former rgb image.
In some embodiments, the system information according to the electronic device judges whether ambient brightness is less than In the preset threshold the step of the following steps are included:
Judge whether the system time falls into predetermined amount of time；With
It is described default to determine that the ambient brightness is less than or equal to when the system time is fallen into the predetermined amount of time Threshold value.
In some embodiments, the system information includes date information, the system according to the electronic device Information judge the step of whether ambient brightness is less than or equal to preset threshold the following steps are included:
The predetermined amount of time is determined according to the date information.
In some embodiments, the system information includes geographical location information, described according to the electronic device System information judge the step of whether ambient brightness is less than or equal to preset threshold the following steps are included:
The predetermined amount of time is determined according to the geographical location information.
In some embodiments, the electronic device includes GPS, and the system information includes the signal strength of GPS.
In some embodiments, the electronic device is described for being communicated with external server to read Weather information System information includes the Weather information.
In some embodiments, the predetermined array includes Bayer array.
In some embodiments, described image pixel unit includes the original pixels of 2*2 array.
In some embodiments, described to be passed through described in the calculating of the first interpolation algorithm according to the pixel value of associated pixel unit The step of pixel value of current pixel the following steps are included:
Calculate the gradual change amount in the associated pixel all directions；
Calculate the weight in the associated pixel all directions；With
According to the pixel value of current pixel described in the gradual change amount and the weight calculation.
In some embodiments, the control method is inserted in the pixel value according to associated pixel unit by first Value-based algorithm calculated before the step of pixel value of the current pixel the following steps are included:
White balance compensation is done to the patch image；
The control method it is described according to the pixel value of associated pixel unit by the first interpolation algorithm calculate described in work as After the step of pixel value of preceding pixel the following steps are included:
White balance compensation reduction is done to the imitative original image.
In some embodiments, the control method is inserted in the pixel value according to associated pixel unit by first Value-based algorithm calculated before the step of pixel value of the current pixel the following steps are included:
Compensating bad point is done to the patch image.
In some embodiments, the control method is inserted in the pixel value according to associated pixel unit by first Value-based algorithm calculated before the step of pixel value of the current pixel the following steps are included:
Crosstalk compensation is done to the patch image.
In some embodiments, the control method is inserted in the pixel value according to associated pixel unit by first Value-based algorithm includes the following steps: after calculating the step of pixel value of the current pixel
Lens shape correction, demosaicing, noise reduction and edge sharpening processing are carried out to the imitative original image.
In some embodiments, the described the step of merging image is converted to merging rgb image includes following step It is rapid:
Using the second interpolation algorithm by the merging image be converted to it is corresponding with the imitative original image go back original image, it is described The complexity of second interpolation algorithm is less than first interpolation algorithm；With
The original image of going back is converted to the merging rgb image.
A kind of control device, for controlling electronic device, the electronic device includes imaging device, the imaging device packet Imaging sensor is included, described image sensor includes sensitive pixel elements array and is arranged on the sensitive pixel elements array Filter unit array, each filter unit array covers the corresponding sensitive pixel elements, each described Sensitive pixel elements include multiple photosensitive pixels, and the control device includes the first control module, judgment module, the first conversion mould Block, the second control module, the second conversion module and third conversion module.
First control module merges image for controlling described image sensor output, and the merging image includes closing And pixel array, multiple photosensitive pixels of the same sensitive pixel elements merge output and are used as a merging pixel.
The judgment module is used to judge whether the ambient brightness is less than according to the system information of the electronic device In preset threshold, the system information includes system time.
First conversion module is used to turn the merging image when the ambient brightness is less than or equal to preset threshold Chemical conversion merges rgb image.
Second control module is used for when the ambient brightness is greater than preset threshold, and control described image sensor is defeated Patch image out, the patch image include the image pixel elements of predetermined array arrangement, and described image pixel unit includes more A original pixels, each corresponding original pixels of the photosensitive pixel.
Second conversion module is used to the patch image being converted to imitative original image, and the imitative original image includes array The imitative original pixel of arrangement, the imitative original pixel includes current pixel, and the original pixels include corresponding with the current pixel Associated pixel.Second conversion module includes the first judging unit, the first computing unit and the second computing unit.
First judging unit be used to judge the color of the current pixel and the associated pixel color whether phase Together.
First computing unit is used for when the color of the current pixel is identical as the color of the associated pixel, will Pixel value of the pixel value of the associated pixel as the current pixel.
Second computing unit is used in the color difference of the color and the associated pixel of the current pixel, root The pixel value of the current pixel, described image pixel list are calculated by the first interpolation algorithm according to the pixel value of associated pixel unit Member include the associated pixel unit, the color of the associated pixel unit it is identical as the current pixel and with the association picture Element is adjacent.
The third conversion module is used to for the imitative original image being converted to imitative former rgb image.
In some embodiments, the judgment module includes second judgment unit and the first determination unit.
The second judgment unit is for judging whether the system time falls into predetermined amount of time.
First determination unit is for determining the environment when the system time is fallen into the predetermined amount of time Brightness is less than or equal to the preset threshold.
In some embodiments, the system information includes date information, and the judgment module includes second determining single Member.
Second determination unit is used to determine the predetermined amount of time according to the date information.
In some embodiments, the judgment module includes third determination unit.
The third determination unit is used to determine the predetermined amount of time according to the geographical location information.
In some embodiments, the electronic device includes GPS, and the system information includes the signal strength of GPS.
In some embodiments, the electronic device is described for being communicated with external server to read Weather information System information includes the Weather information.
In some embodiments, the predetermined array includes Bayer array.
In some embodiments, described image pixel unit includes the original pixels of 2*2 array.
In some embodiments, second computing unit include the first computation subunit, the second computation subunit and Third computation subunit.
First computation subunit is used to calculate the gradual change amount in the associated pixel all directions.
Second computation subunit is used to calculate the weight in the associated pixel all directions.
The third computation subunit is used for the pixel of the current pixel according to the gradual change amount and the weight calculation Value.
In some embodiments, second conversion module includes that white balance compensation unit and white balance compensation reduction are single Member.
The white balance compensation unit is for doing white balance compensation to the patch image.
The white balance compensation reduction unit is for doing white balance compensation reduction to the imitative original image.
In some embodiments, second conversion module includes compensating bad point unit.
The compensating bad point unit is for doing compensating bad point to the patch image.
In some embodiments, second conversion module includes crosstalk compensation unit.
The crosstalk compensation unit is for doing crosstalk compensation to the patch image.
In some embodiments, second conversion module includes processing unit.
The processing unit is used to carry out lens shape correction, demosaicing, noise reduction and edge to the imitative original image sharp Change processing.
In some embodiments, first conversion module includes the first conversion unit and the second conversion unit.
The merging image for being converted to and the imitative original image by first conversion unit using the second interpolation algorithm Original image is gone back as corresponding, the complexity of second interpolation algorithm is less than first interpolation algorithm.
Second conversion unit is used to the original image of going back being converted to the merging rgb image.
A kind of electronic device includes imaging device and the control device.
In some embodiments, the electronic device includes mobile phone or tablet computer.
In some embodiments, the imaging device includes Front camera or rearmounted camera.
Control method, control device and electronic device of the invention using electronic device system information to ambient brightness into Row judgement, control imaging sensor export suitable image, avoid the fixed outputting high quality image bring of imaging sensor big Work is measured, to reduce electronic device functions duration, working efficiency is improved, improves user satisfaction.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention is from combining in description of the following accompanying drawings to embodiment by change It obtains obviously and is readily appreciated that, in which:
Fig. 1 is the flow diagram of the control method of embodiment of the present invention.
Fig. 2 is another flow diagram of the control method of embodiment of the present invention.
Fig. 3 is the functional block diagram of the control device of embodiment of the present invention.
Fig. 4 is the functional block diagram of the second conversion module of embodiment of the present invention.
Fig. 5 is the functional block diagram of the electronic device of embodiment of the present invention.
Fig. 6 is the flow diagram of the control method of certain embodiments of the present invention.
Fig. 7 is the functional block diagram of the judgment module of certain embodiments of the present invention.
Fig. 8 is the flow diagram of the control method of certain embodiments of the present invention.
Fig. 9 is the functional block diagram of the judgment module of certain embodiments of the present invention.
Figure 10 is the flow diagram of the control method of certain embodiments of the present invention.
Figure 11 is the functional block diagram of the judgment module of certain embodiments of the present invention.
Figure 12 is the module diagram of the imaging sensor of embodiment of the present invention.
Figure 13 is the circuit diagram of the imaging sensor of embodiment of the present invention.
Figure 14 is the schematic diagram of the filter unit of embodiment of the present invention
Figure 15 is the structural schematic diagram of the imaging sensor of embodiment of the present invention.
Figure 16 is the merging image state schematic diagram of embodiment of the present invention.
Figure 17 is the status diagram of the patch image of embodiment of the present invention.
Figure 18 is the status diagram of the control method of embodiment of the present invention.
Figure 19 is the flow diagram of the control method of certain embodiments of the present invention.
Figure 20 is the functional block diagram of the second computing unit of certain embodiments of the present invention.
Figure 21 is the flow diagram of the control method of certain embodiments of the present invention.
Figure 22 is the functional block diagram of the second conversion module of certain embodiments of the present invention.
Figure 23 is the image pixel elements schematic diagram of the patch image of certain embodiments of the present invention
Figure 24 is the flow diagram of the control method of certain embodiments of the present invention.
Figure 25 is the functional block diagram of the second conversion module of certain embodiments of the present invention.
Figure 26 is the flow diagram of the control method of certain embodiments of the present invention.
Figure 27 is the functional block diagram of the first conversion module of certain embodiments of the present invention.
Main element symbol description:
Electronic device 1000, imaging device 100, imaging sensor 10, sensitive pixel elements array 12, sensitive pixel elements 12a, photosensitive pixel subelement 120, photosensitive pixel 122, sensor devices 1222, transfer tube 1224, source follower 124, modulus Converter 126, adder 128, filter unit array 14, filter unit 14a, control device 200, the first control module 211, judgment module 214, the first conversion module 216, the second control module 218, the second conversion module 220, third conversion module 230, the first judging unit 221, the first computing unit 222, the second computing unit 223, second judgment unit 2142, first determine Unit 2144, the second determination unit 2146, third determination unit 2148, the first computation subunit 2232, the second computation subunit 2234, third computation subunit 2236, white balance compensation unit 224, white balance compensation reduction unit 225, compensating bad point unit 226, crosstalk compensation unit 227, processing unit 228, the first conversion unit 2162, the second conversion unit 2164.
Specific embodiment
Embodiments of the present invention are described below in detail, the embodiment of the embodiment is shown in the accompanying drawings, wherein Same or similar label indicates same or similar element or element with the same or similar functions from beginning to end.Lead to below It crosses the embodiment being described with reference to the drawings to be exemplary, for explaining only the invention, and should not be understood as to limit of the invention System.
Referring to Fig. 1, the control method of embodiment of the present invention, for controlling electronic device 1000, electronic device 1000 Including imaging device 100, imaging device 100 includes imaging sensor 10, and imaging sensor 10 includes sensitive pixel elements array 12 and the filter unit array 14 that is arranged on sensitive pixel elements array 12, each covering of filter unit array 14 correspond to One sensitive pixel elements 12a, each sensitive pixel elements 12a include multiple photosensitive pixels 122, and control method includes following step It is rapid:
S211: control imaging sensor output merges image, and merging image includes merging pixel array, same photosensitive pixel Multiple photosensitive pixels of unit merge output and are used as a merging pixel；
S214: judge whether ambient brightness is less than or equal to preset threshold, system information according to the system information of electronic device Including system time；
S216: when ambient brightness is less than or equal to preset threshold, image will be merged and be converted to merging rgb image；
S218: when ambient brightness is greater than preset threshold, control imaging sensor exports patch image, and patch image includes The image pixel elements of predetermined array arrangement, image pixel elements include multiple original pixels, and each photosensitive pixel is one corresponding Original pixels；
S220: being converted to imitative original image for patch image, and imitative original image includes the imitative original pixel of array arrangement, imitates original pixel Including current pixel, original pixels include associated pixel corresponding with current pixel；With
S230: imitative original image is converted to imitative former rgb image.
Referring to Fig. 2, in some embodiments, step S220 the following steps are included:
S221: whether the color of the color and associated pixel that judge current pixel is identical；
S222: in the color of current pixel and the identical color of associated pixel, using the pixel value of associated pixel as working as The pixel value of preceding pixel；With
S223: current pixel color and associated pixel color difference when, according to the pixel value of associated pixel unit The pixel value of current pixel is calculated by the first interpolation algorithm, image pixel elements include associated pixel unit, associated pixel list The color of member is identical and adjacent with associated pixel as current pixel.
Please refer to Fig. 3 and Fig. 4, the control device 200 of embodiment of the present invention, for controlling electronic device 1000.Electronics Device 1000 includes imaging device 100, and imaging device 100 includes imaging sensor 10, and imaging sensor 10 includes photosensitive pixel Cell array 12 and the filter unit array 14 being arranged on sensitive pixel elements array 12, each filter unit array 14 Covering corresponding sensitive pixel elements a 12a, each sensitive pixel elements 12a includes multiple photosensitive pixels 122, control device 200 include the first control module 211, judgment module 214, the first conversion module 216, the second control module 218, second conversion mould Block 220 and third conversion module 230.First control module 211 merges image for controlling the output of imaging sensor 10, merges figure As including merging pixel array, multiple photosensitive pixels 122 of same sensitive pixel elements 12a merge output and are used as a merging picture Element.Judgment module 214 is used to according to the system information of electronic device 1000 judge whether ambient brightness is less than or equal to preset threshold, System information includes system time.First conversion module 216 is used to scheme when ambient brightness is less than or equal to preset threshold by merging As being converted to merging rgb image.Second control module 218 is used to control image sensing when ambient brightness is greater than preset threshold Device 10 exports patch image, and patch image includes the image pixel elements of predetermined array arrangement, and image pixel elements include multiple Original pixels, the corresponding original pixels of each photosensitive pixel 122.Second conversion module 220 is for patch image to be converted to Imitative original image, imitative original image includes the imitative original pixel of array arrangement, and imitative original pixel includes current pixel, and original pixels include and work as The corresponding associated pixel of preceding pixel, the second conversion module 220 include the first judging unit 221, the first computing unit 222 and second Computing unit 223.First judging unit 221 be used for judge current pixel color and associated pixel color it is whether identical.The One computing unit 222 is used in the color of current pixel and the identical color of associated pixel, and the pixel value of associated pixel is made For the pixel value of current pixel.Second computing unit 223 be used for current pixel color and associated pixel color difference when, The pixel value of current pixel is calculated by the first interpolation algorithm according to the pixel value of associated pixel unit, image pixel elements include Associated pixel unit, the color of associated pixel unit are identical and adjacent with associated pixel as current pixel.Third conversion module 230 For imitative original image to be converted to imitative former rgb image.
In other words, the control method of embodiment of the present invention can be real by the control device 200 of embodiment of the present invention It is existing, wherein step S211 can be realized that step S214 can be realized by judgment module 214, step by the first control module 211 S216 can be realized that step S218 can be realized by the second control module 218 by the first conversion module 216, and step S220 can be with It is realized by the second conversion module 220, step S230 can be realized by third conversion module 230.
Referring to Fig. 5, the electronic device 1000 of embodiment of the present invention includes imaging device 100 and embodiment of the present invention Control device 200, in other words the control device 200 of embodiment of the present invention can be applied to the electricity of embodiment of the present invention Sub-device 1000.
Control method, control device 200 and electronic device 1000 of the invention utilizes the system information of electronic device 1000 Ambient brightness is judged, control imaging sensor 10 exports suitable image, avoids the fixed output of imaging sensor 10 high Quality image bring extensive work improves working efficiency to reduce 1000 operating time of electronic device, and it is satisfied to improve user Degree.
In some embodiments, the electronics that electronic device 1000 has imaging device including mobile phone or tablet computer etc. is set It is standby, any restrictions are not done herein.The electronic device 1000 of embodiment of the present invention is mobile phone.
In some embodiments, imaging device 100 includes Front camera or rearmounted camera.
In some embodiments, in such a way that system information obtains ambient brightness with ambient brightness is obtained than other With advantage.Such as lead to Fibre Optical Sensor since the visual field of light sensor is limited using light sensor acquisition ambient brightness Device can only get the light information in visual field, cause and actual environment brightness generates deviation, obtain environment using system information The available more comprehensive environmental information of brightness.There are also the methods for example, by using image procossing to obtain ambient brightness, at image Reason needs to take a substantial amount of time carry out complex calculation, and may malfunction in image processing process, when ambient brightness obtains Between it is long, and consume the hardware resource of electronic device 1000, obtaining ambient brightness using system information can be to avoid complicated fortune It calculates.
Referring to Fig. 6, in some embodiments, step S214 the following steps are included:
S2142: judge whether system time falls into predetermined amount of time；With
S2144: determine that ambient brightness is less than or equal to preset threshold when system time is fallen into predetermined amount of time.
Referring to Fig. 7, in some embodiments, judgment module 214 includes that second judgment unit 2142 and first determine Unit 2144.Second judgment unit 2142 is for judging whether system time falls into predetermined amount of time.First determination unit 2144 for determining that ambient brightness is less than or equal to preset threshold when system time is fallen into predetermined amount of time.
In other words, step S2142 can be realized by second judgment unit 2142, and step S2144 can be determined by first Unit 2144 is realized.
It is appreciated that predetermined amount of time can be real ambient light line it is poor when, such as 6 pm to next day 6:00 AM, when system time falls into predetermined amount of time, it can be determined that ambient brightness is low at this time.In the case where ambient brightness, It can control the output of imaging sensor 10 and merge image, the acquisition energy of light is improved using the merging output of multiple photosensitive pixels 122 Power.
Referring to Fig. 8, in some embodiments, system information includes date information, step S214 the following steps are included:
S2146: predetermined amount of time is determined according to date information.
Referring to Fig. 9, in some embodiments, system information includes date information, judgment module 214 includes second true Order member 2146.Second determination unit 2146 is used to determine predetermined amount of time according to date information.
In other words, step S2146 can be realized by the second determination unit 2146.
It is appreciated that since the daytime of same date and the period at night be not different, that is to say, that with the period Ambient brightness can change according to not same date, so when determining predetermined amount of time, it can be according to different date informations It is determined.Such as when the Spring Equinox, day and night divide equally, can primarily determine that predetermined amount of time includes morning 6 pm to next day Upper 6 points；Such as when the Summer Solstice, the time on daytime be it is longest, can primarily determine that predetermined amount of time includes at 7 points in afternoon To 5 points of next day.
In this way, can determine the predetermined amount of time for being more in line with actual conditions according to different system times.
Referring to Fig. 10, in some embodiments, system information includes geographical location information, step S214 includes following Step:
S2148: predetermined amount of time is determined according to geographical location information.
Figure 11 is please referred to, in some embodiments, judgment module 214 includes third determination unit 2148.Third determines Unit 2148 is used to determine predetermined amount of time according to geographical location information.
It is appreciated that the predetermined amount of time of diverse geographic location be also possible to it is different, with geographical locations such as latitudes The period at difference, daytime and night is it can also happen that variation.Such as the case where the South Pole will appear polar day polar night, cause one whole It ambient brightness is all very high or all very low.Therefore, predetermined amount of time can be adjusted according to the difference in geographical location.
In some embodiments, electronic device 1000 includes GPS, and system information includes the signal strength of GPS.
In this way, can use the geographical location that GPS obtains electronic device 1000, to adjust pre- timing according to geographical location Between section so that imaging device 100 can obtain better working effect.
In some embodiments, electronic device 1000 with external server for communicating to read Weather information, system Information includes Weather information.
It is appreciated that Weather information can also influence the selection of predetermined amount of time.Such as when rainy day, environment is bright Degree is whole lower.
Also referring to Figure 12 to Figure 15, the imaging sensor 10 of embodiment of the present invention includes sensitive pixel elements array 12 and the filter unit array 14 that is arranged on sensitive pixel elements array 12.
Further, sensitive pixel elements array 12 includes multiple sensitive pixel elements 12a, each sensitive pixel elements 12a includes multiple adjacent photosensitive pixels 122.Each photosensitive pixel 122 includes a sensor devices 1222 and a transfer tube 1224, wherein sensor devices 1222 can be photodiode, and transfer tube 1224 can be MOS transistor.
Filter unit array 14 includes multiple filter unit 14a, each corresponding sense of filter unit 14a covering Light pixel unit 12a.
Specifically, in some examples, filter unit array 14 includes Bayer array, in other words, adjacent four Filter unit 14a is respectively a red filter blade unit, a blue filter blade unit and two green filter blade units.
Each sensitive pixel elements 12a corresponds to the filter unit 14a of same color, if a sensitive pixel elements It altogether include n adjacent sensor devices 1222 in 12a, then a filter unit 14a covers a sensitive pixel elements N sensor devices 1222 in 12a, filter unit 14a can be Construction integration, can also be by n independent sub- optical filterings Piece assembly and connection is together.
In some embodiments, each sensitive pixel elements 12a includes four adjacent photosensitive pixels 122, and adjacent two A photosensitive pixel 122 collectively forms a photosensitive pixel subelement 120, photosensitive pixel subelement 120 further include a source electrode with With device 124 and an analog-digital converter 126.Sensitive pixel elements 12a further includes an adder 128.Wherein, a light-sensitive image One termination electrode of each transfer tube 1224 in sub-prime unit 120 is connected to the cathode electrode of corresponding sensor devices 1222, often The other end of a transfer tube 1224 is commonly connected to the gate electrode of source follower 124, and passes through 124 source of source follower Pole electrode is connected to an analog-digital converter 126.Wherein, source follower 124 can be MOS transistor.Two photosensitive pixels Subelement 120 is connected to adder 128 by respective source follower 124 and analog-digital converter 126.
In other words, adjacent four in a sensitive pixel elements 12a of the imaging sensor 10 of embodiment of the present invention A sensor devices 1222 share a filter unit 14a with color, and each sensor devices 1222 are correspondingly connected with a transmission Pipe 1224, the shared source follower 124 of two neighboring sensor devices 1222 and an analog-digital converter 126, adjacent four A sensor devices 1222 share an adder 128.
Further, four adjacent sensor devices 1222 are in 2*2 array arrangement.Wherein, a photosensitive pixel subelement Two sensor devices 1222 in 120 may be at same row.
In imaging, when the two photosensitive pixel subelements 120 covered under same filter unit 14a, four senses in other words When optical device 1222 exposes simultaneously, exportable merging image in turn can be merged to pixel.
Specifically, sensor devices 1222 are used to convert illumination to charge, and the charge generated is proportional to intensity of illumination Relationship, transfer tube 1224 are used for according to control signal being turned on or off come control circuit.When circuit conducting, source follower 124 for converting voltage signal for the charge signal that sensor devices 1222 are generated through illumination.Analog-digital converter 126 is used for voltage Signal is converted to digital signal.Adder 128 is used to for two ways of digital signals to be added common output.
Figure 16 is please referred to, for example with the imaging sensor 10 of 16M, the imaging sensor 10 of embodiment of the present invention can The photosensitive pixel 122 of 16M is merged into 4M, in other words, output merges image, after merging, the sizableness of photosensitive pixel 122 In becoming 4 times of original size, to improve the sensitivity of photosensitive pixel 122.Further, since in imaging sensor 10 It is all random noise that noise is most of, for the photosensitive pixel 122 before merging, it is possible to wherein one or two picture There are noises in element, and after four photosensitive pixels 122 are merged into a big photosensitive pixel 122, noise is reduced to this The influence of big pixel, that is to say and reduce noise, improve signal-to-noise ratio.
But while 122 size of photosensitive pixel becomes larger, since pixel value reduces, the resolution for merging image also will drop It is low.
It, can be defeated when four sensor devices 1222 covered under same filter unit 14a successively expose in imaging Patch image out.
Specifically, sensor devices 1222 are used to convert illumination to charge, and the charge generated is proportional to intensity of illumination Relationship, transfer tube 1224 are used for according to control signal being turned on or off come control circuit.When circuit conducting, source follower 124 for converting voltage signal for the charge signal that sensor devices 1222 are generated through illumination.Analog-digital converter 126 is used for voltage Signal is converted to digital signal.
Figure 17 is please referred to, for example with the imaging sensor 10 of 16M, the imaging sensor 200 of embodiment of the present invention The photosensitive pixel 122 of 16M can also be kept to export, export patch image in other words, patch image includes image pixel elements, figure As pixel unit includes the original pixels of 2*2 array arrangement, the size of the original pixels is identical as 122 size of photosensitive pixel, so And the filter unit 14a due to covering adjacent four sensor devices 1222 is same color, is also said immediately, although four photosensitive Device 1222 exposes respectively, but the filter unit 14a color for covering it is identical, therefore, each image pixel elements of output Adjacent four original pixels colors it is identical, still can not improve the resolution of image.
The control method of embodiment of the present invention can be used for handling the patch image of output, to obtain imitative original Image.
It is appreciated that merging image in output, four adjacent homochromy photosensitive pixels 122 are to merge pixel output, such as This, the four adjacent merging pixels merged in image are still considered as typical Bayer array, can direct controlled device 200 processing are to export merging rgb image.And patch image each photosensitive pixel 122 in output exports respectively, due to adjacent Four 122 colors of photosensitive pixel are identical, and therefore, the color of four adjacent original pixels of an image pixel elements is identical, are Atypical Bayer array.And control device 200 can not directly be handled atypia Bayer array, in other words, schemed When as sensor 10 using same control device 200, the rgb image output for compatible both of which is the conjunction under merging patterns And the imitative former rgb image output under rgb image output and color lump mode, patch image need to be converted to imitative original image, or Say that the image pixel elements by atypia Bayer array are converted into the pixel arrangement of typical Bayer array.
Imitative original image includes the imitative original pixel in Bayer array arrangement.Imitative original pixel includes current pixel, original pixels packet Include associated pixel corresponding with current pixel.
Figure 18 is please referred to, by taking Figure 18 as an example, current pixel R3 ' 3 ' and R5 ' 5 ', corresponding associated pixel is respectively R33 And B55.
When obtaining current pixel R3 ' 3 ', since R3 ' 3 ' is identical as the color of corresponding associated pixel R33, turning Directly using the pixel value of R33 as the pixel value of R3 ' 3 ' when change.
When obtaining current pixel R5 ' 5 ', since R5 ' 5 ' and the color of corresponding associated pixel B55 be not identical, it is clear that no Can be directly using the pixel value of B55 as the pixel value of R5 ' 5 ', the side that needs to pass through interpolation according to the associated pixel unit of R5 ' 5 ' Formula is calculated.
It should be noted that the color attribute numerical value that pixel value above and hereinafter is construed broadly to mean as the pixel, Such as color-values.
Associated pixel unit include it is multiple, such as 4, color image identical as current pixel and adjacent with associated pixel Original pixels in pixel unit.
It should be noted that adjacent herein shall be understood in a broad sense, by taking Figure 18 as an example, the corresponding associated pixel of R5 ' 5 ' is B55, the image slices where associated pixel unit adjacent with the image pixel elements where B55 and identical with 5 ' color of R5 ' Plain unit is respectively the image pixel elements where R44, R74, R47, R77, and the not spatially image where distance B55 The farther other red image pixels units of pixel unit.Wherein, with B55 spatially apart from nearest red original pixels Respectively R44, R74, R47 and R77, in other words, the associated pixel unit of R5 ' 5 ' are made of R44, R74, R47 and R77, R5 ' 5 ' is identical as the color of R44, R74, R47 and R77, associated pixel unit R 44, R74, R47 and R77 B55 phase corresponding with R5 ' 5 ' It is adjacent.
In this way, be directed to different situations current pixel, in different ways convert original pixels to imitative original pixel, from And imitative original image is converted by patch image, when due to shooting image, using the optical filter of special Bayer array structure, mention High signal noise ratio (snr) of image, and during image processing, interpolation processing is carried out to patch image by interpolation algorithm, is improved The resolution ratio and resolution of image.
Please refer to Figure 19, in some embodiments, step S223 the following steps are included:
S2232: the gradual change amount in associated pixel unit all directions is calculated；
S2234: the weight in associated pixel unit all directions is calculated；With
S2236: according to gradual change amount and the pixel value of weight calculation current pixel.
Figure 20 is please referred to, in some embodiments, the second computing unit 223 includes the first computation subunit 2232, the Two computation subunits 2234 and third computation subunit 2236.Step S2232 can be realized by the first computation subunit 2232, be walked Rapid S2234 can be realized that step S2236 can be realized by third computation subunit 2236 by the second computation subunit 2234.Or Person says that the first computation subunit 2232 is used to calculate the gradual change amount in associated pixel unit all directions, the second computation subunit 2234 for calculating the weight in associated pixel unit all directions, and third computation subunit 2236 is used for according to gradual change amount and power The pixel value of re-computation current pixel.
Specifically, interpolation processing mode is the energy gradual change of reference picture in different directions, will be corresponding with current pixel Identical and adjacent with the associated pixel associated pixel unit of color according to gradual change weight size in different directions, pass through line The pixel value of current pixel is calculated in the mode of property interpolation.Wherein, on the lesser direction of energy variation amount, reference specific gravity compared with Greatly, therefore, the weight in interpolation calculation is larger.
In some examples, for convenience of calculating, only consider both horizontally and vertically.
R5 ' 5 ' is obtained by R44, R74, R47 and R77 interpolation, and in the horizontal and vertical directions and there is no color it is identical Original pixels, therefore need the first component that the color in the horizontal and vertical directions is calculated according to associated pixel unit.Wherein, water Square upward component is R45 and R75, vertically-oriented component are R54 and R57 can pass through R44, R74, R47 and R77 respectively It is calculated.
Specifically, R45=R44*2/3+R47*1/3, R75=2/3*R74+1/3*R77, R54=2/3*R44+1/3* R74, R57=2/3*R47+1/3*R77.
Then, it calculates separately in gradual change amount and weight both horizontally and vertically, in other words, according to the color in difference The gradual change amount in direction, to determine the reference weight of the different directions in interpolation, in the small direction of gradual change amount, weight is larger, and The biggish direction of gradual change amount, weight are smaller.Wherein, gradual change amount X1=in the horizontal direction | R45-R75 |, in vertical direction Gradual change amount X2=| R54-R57 |, W1=X1/ (X1+X2), W2=X2/ (X1+X2).
In this way, being calculated according to above-mentioned, R5 ' 5 '=(2/3*R45+1/3*R75) * W2+ (2/3*R54+1/3* R57)*W1.If being appreciated that, X1 is greater than X2, and W1 is greater than W2, therefore the weight of horizontal direction is W2 when calculating, and Vertical Square To weight be W1, vice versa.
In this way, the pixel value of current pixel can be calculated according to interpolation algorithm.According to the above-mentioned processing to associated pixel Mode can convert original pixels to the imitative original pixel in typical Bayer array arrangement, in other words, 2*2 gusts of adjacent four The imitative original pixel of column includes that a red imitates original pixel, and two greens imitate original pixel and a blue imitates original pixel.
It should be noted that the mode of interpolation includes but is not limited to only to consider vertically disclosed in the present embodiment when calculating With the mode of the pixel value of horizontal both direction same color, such as can also refer to other colors pixel value.
Please refer to Figure 21 and Figure 22, in some embodiments, before step S223 comprising steps of
S224: white balance compensation is done to patch image；
After step S223 comprising steps of
S225: white balance compensation reduction is done to imitative original image.
In some embodiments, the second conversion module 220 includes that white balance compensation unit 224 and white balance compensation restore Unit 225.Step S224 can be realized that step S225 can be by white balance compensation reduction unit by white balance compensation unit 224 225 realize.In other words, white balance compensation unit 224 is for doing white balance compensation, white balance compensation reduction unit to patch image 225 for doing white balance compensation reduction to imitative original image.
Specifically, in some instances, during converting imitative original image for patch image, in Interpolation Process, The imitative original pixel of red and blue often not only with reference to channel identical with its color original pixels color, can also be with reference to green Therefore the color weight of the original pixels in channel needs to carry out white balance compensation before interpolation, white to exclude in interpolation calculation The influence of balance.In order not to destroy the white balance of patch image, therefore, need imitative original image carrying out white balance after interpolation Compensation reduction is restored according to red, green and blue yield value in the compensation when reduction.
In this way, the influence of the white balance in Interpolation Process can be excluded, and enable to the imitative original image obtained after interpolation Keep the white balance of patch image.
Referring to Figure 21 and Figure 22, in some embodiments, before step S223 comprising steps of
S226: compensating bad point is done to patch image.
In some embodiments, the second conversion module 220 includes compensating bad point unit 226.Step S226 can be by bad Point compensating unit 226 is realized.In other words, compensating bad point unit 226 is for doing compensating bad point to patch image.
It is appreciated that being limited to manufacturing process, imaging sensor 10 may have bad point, and bad point is not usually with sensitivity Change and same color is presented always, the presence of bad point will affect picture quality, therefore, to guarantee the accurate of interpolation, not by bad The influence of point, needs to carry out compensating bad point before interpolation.
Specifically, during compensating bad point, original pixels can be detected, when detect a certain original pixels be it is bad When point, compensating bad point can be carried out according to the pixel value of other original images of the image pixel elements where it.
In this way, influence of the bad point to interpolation processing can be excluded, picture quality is improved.
Referring to Figure 21 and Figure 22, in some embodiments, before step S223 comprising steps of
S227: crosstalk compensation is done to patch image.
In some embodiments, the second conversion module 220 includes crosstalk compensation unit 227.Step S227 can be by going here and there Disturb the realization of compensating unit 227.In other words, crosstalk compensation unit 227 is for doing crosstalk compensation to patch image.
Specifically, the optical filter of four photosensitive pixels 122 covering same color in a sensitive pixel elements 12a, and There may be the differences of sensitivity between photosensitive pixel 122, so as to imitate in the imitative former rgb image of original image conversion output Solid color regions will appear fixed spectral noise, influence the quality of image.Therefore, it is necessary to carry out crosstalk compensation to patch image.
In some embodiments, setting compensating parameter is the following steps are included: provide predetermined luminous environment；Imaging device is set Imaging parameters；Shoot multi-frame images；Multiple image is handled to obtain crosstalk compensation parameter；It is stored in by crosstalk compensation parameter In described image processing unit.
Such as above-mentioned explanation, crosstalk compensation is carried out, is needed in 10 manufacturing process of imaging sensor of imaging device 100 Compensating parameter is obtained, and the relevant parameter of crosstalk compensation is preset in the memory of imaging device 100 or is installed imaging device In 100 electronic device 1000 such as mobile phone or tablet computer.
Predetermined luminous environment for example may include LED uniform light board, the colour temperature of 5000K or so, 1000 lux of brightness or so, imaging Parameter may include yield value, fast gate value and lens location.After setting relevant parameter, the acquisition of crosstalk compensation parameter is carried out.
In treatment process, imaging parameters first in the luminous environment of setting to set obtain multiple patch images, and It is merged into a patch image, can so reduce the influence of noise using individual patch image as calibration basis.
Figure 23 is please referred to, by taking the image pixel elements Gr in Figure 23 as an example comprising Gr1, Gr2, Gr3 and Gr4, crosstalk are mended It repays purpose and is that there may be the photosensitive pixels of difference by compensating into originally calibrating to same level by sensitivity.The image pixel The average pixel value of unit is Gr_avg=(Gr1+Gr2+Gr3+Gr4)/4, can characterize the photosensitive of this four photosensitive pixels substantially The average level of degree is worth based on this average value, calculates separately Gr1/Gr_avg, Gr2/Gr_avg, Gr3/Gr_avg and Gr4/Gr_avg, it will be understood that by calculating the pixel value of each original pixels and the mean pixel of the image pixel elements The ratio of value can reflect the deviation of each original pixels and basic value substantially, record four ratios and remember as compensating parameter It records in the memory of relevant apparatus, each original pixels is compensated with being transferred in imaging, so that crosstalk is reduced, Improve picture quality.
In general, whether should also verify set parameter after setting crosstalk compensation parameter accurate.
In verification process, a patch image is obtained with identical luminous environment and imaging parameters first, foundation is calculated Compensating parameter to the patch image carry out crosstalk compensation, calculate compensated Gr ' _ avg, Gr ' 1/Gr ' _ avg, Gr ' 2/Gr ' _ Avg, Gr ' 3/Gr ' _ avg and Gr ' 4/Gr ' _ avg.Judge whether compensating parameter is accurate according to calculated result, judgement can be according to macro It sees and considers with microcosmic two angles.It is microcosmic to refer to some original pixels still deviation is larger after compensation, easily made after imaging User's perception, and it is macroscopical then from global angle, when that is to say more there are still the total number of the original pixels of deviation after compensation, Even if individually the deviation of each original pixels is little at this time, but still can be perceived as a whole.Therefore, should Setting is directed to one proportion threshold value of microcosmic setting, and a proportion threshold value and an amount threshold need to be arranged for macroscopic view.Such as This, can verify the crosstalk compensation parameter of setting, it is ensured that compensating parameter it is correct, to reduce crosstalk to the shadow of picture quality It rings.
Figure 24 and Figure 25 are please referred to, in some embodiments, is further comprised the steps of: after step S223
S228: eyeglass shadow correction, demosaicing, noise reduction and edge sharpening are carried out to imitative original image and handled.
In some embodiments, the second conversion module 220 includes processing unit 228.Step S228 can be single by processing Member 228 realizes that in other words, processing unit 228 is used to carry out eyeglass shadow correction, demosaicing, noise reduction and side to imitative original image Edge Edge contrast.
It is appreciated that imitating original pixel arrangement is typical Bayer array after converting imitative original image for patch image, can adopt It is handled with processing unit 200, includes that eyeglass shadow correction, demosaicing, noise reduction and edge sharpening are handled in treatment process, It exports in this way, imitative former rgb image can be obtained after processing to user.
Please refer to Figure 26 and Figure 27, in some embodiments, step S216 the following steps are included:
S2162: using the second interpolation algorithm will merge image be converted to it is corresponding with imitative original image go back original image, second inserts The complexity of value-based algorithm is less than the first interpolation algorithm；With
S2164: original image will be gone back and be converted to merging rgb image.
In some embodiments, the first conversion module 216 includes the first conversion unit 2162 and the second conversion unit 2164.First conversion unit 2162 is used to be converted to reduction corresponding with imitative original image for image is merged using the second interpolation algorithm Image, the complexity of the second interpolation algorithm is less than the first interpolation algorithm.Second conversion unit 2164 will be for that will go back original image conversion At merging rgb image.In other words, step S2162 can be realized by the first conversion unit 2162, and step S2164 can be by Two conversion units 2164 are realized.
In some embodiments, the time complexity of the second interpolation algorithm and space complexity are all than the first interpolation algorithm It is small.The complexity of algorithm includes time complexity and space complexity, time complexity be used to that metric algorithm needs to expend when Between, space complexity is the memory space for needing to expend for metric algorithm.Time complexity is small to illustrate what algorithm needed to expend Time is few, and the small memory space for illustrating that algorithm needs to expend of space complexity is small, therefore, is conducive to mention using the second interpolation algorithm High arithmetic speed improves user experience so that process of taking pictures is more smooth.
In some embodiments, the second interpolation algorithm, which can be, will simply merge image quadruplication without passing through Other complex calculations are crossed, to obtain corresponding with imitative original image going back original image.
It is appreciated that obtaining imitative former rgb image by the first conversion unit 2162 and the conversion of the second conversion unit 2164 Afterwards, noise reduction can be carried out to imitative former rgb image and edge sharpening is handled, in this way, the imitative original of better quality can be obtained after processing Rgb image is exported to user.
In the description of embodiments of the present invention, it is to be understood that term " center ", " longitudinal direction ", " transverse direction ", " length Degree ", " width ", " thickness ", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", The orientation or positional relationship of the instructions such as "outside", " clockwise ", " counterclockwise " is to be based on the orientation or positional relationship shown in the drawings, only It is embodiments of the present invention and simplified description for ease of description, rather than the device or element of indication or suggestion meaning are necessary It with specific orientation, is constructed and operated in a specific orientation, therefore should not be understood as the limitation to embodiments of the present invention.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or Implicitly include one or more feature.In the description of embodiments of the present invention, the meaning of " plurality " is two It is a or more than two, unless otherwise specifically defined.
In the description of embodiments of the present invention, it should be noted that unless otherwise clearly defined and limited, term " installation ", " connected ", " connection " shall be understood in a broad sense, for example, it may be fixedly connected, may be a detachable connection or one Connect to body；It can be mechanical connection, be also possible to be electrically connected or can mutually communicate；It can be directly connected, can also lead to It crosses intermediary to be indirectly connected, can be the connection inside two elements or the interaction relationship of two elements.For ability For the those of ordinary skill in domain, can understand as the case may be above-mentioned term in embodiments of the present invention specifically contain Justice.
In embodiments of the present invention unless specifically defined or limited otherwise, fisrt feature second feature it "upper" or "lower" may include that the first and second features directly contact, may include the first and second features be not directly to connect yet It touches but by the other characterisation contact between them.Moreover, fisrt feature second feature " on ", " top " and " on Face " includes fisrt feature right above second feature and oblique upper, or to be merely representative of first feature horizontal height special higher than second Sign.Fisrt feature include under the second feature " below ", " below " and " below " fisrt feature immediately below second feature and obliquely downward Side, or first feature horizontal height is merely representative of less than second feature.
Above disclosure provides many different embodiments or example is used to realize embodiments of the present invention not Same structure.In order to simplify the disclosure of embodiments of the present invention, above the component of specific examples and setting are described.When So, they are merely examples, and is not intended to limit the present invention.In addition, embodiments of the present invention can be in different examples Repeat reference numerals and/or reference letter in son, this repetition are for purposes of simplicity and clarity, itself not indicate to be begged for By the relationship between various embodiments and/or setting.In addition, the various specific techniques that embodiments of the present invention provide With the example of material, but those of ordinary skill in the art may be aware that the application of other techniques and/or other materials make With.
In the description of this specification, reference term " embodiment ", " some embodiments ", " schematically implementation The description of mode ", " example ", specific examples or " some examples " etc. means the tool described in conjunction with the embodiment or example Body characteristics, structure, material or feature are contained at least one embodiment or example of the invention.In the present specification, Schematic expression of the above terms are not necessarily referring to identical embodiment or example.Moreover, the specific features of description, knot Structure, material or feature can be combined in any suitable manner in any one or more embodiments or example.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for Instruction execution system, device or equipment (such as computer based system, including the system of processing module or other can be from instruction Execute system, device or equipment instruction fetch and the system that executes instruction) use, or combine these instruction execution systems, device or Equipment and use.For the purpose of this specification, " computer-readable medium " can be it is any may include, store, communicating, propagating or Transfer program uses for instruction execution system, device or equipment or in conjunction with these instruction execution systems, device or equipment Device.The more specific example (non-exhaustive list) of computer-readable medium include the following: there are one or more wirings Electrical connection section (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory (ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of embodiments of the present invention can be with hardware, software, firmware or their combination come real It is existing.In the above-described embodiment, multiple steps or method can be with storages in memory and by suitable instruction execution system The software or firmware of execution is realized.For example, if realized with hardware, in another embodiment, ability can be used Any one of following technology or their combination well known to domain is realized: being had for realizing logic function to data-signal The discrete logic of logic gates, the specific integrated circuit with suitable combinational logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
In addition, each functional unit in various embodiments of the present invention can integrate in a processing module, it can also To be that each unit physically exists alone, can also be integrated in two or more units in a module.It is above-mentioned integrated Module both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module If in the form of software function module realize and when sold or used as an independent product, also can store one calculating In machine read/write memory medium.
Storage medium mentioned above can be read-only memory, disk or CD etc..
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example Property, it is not considered as limiting the invention, those skilled in the art within the scope of the invention can be to above-mentioned Embodiment is changed, modifies, replacement and variant.

Claims (27)

1. a kind of control method, for controlling electronic device, which is characterized in that the electronic device includes imaging device, described Imaging device includes imaging sensor, and described image sensor includes sensitive pixel elements array and is arranged in the photosensitive pixel Filter unit array in cell array, the filter unit covering corresponding one of each of described filter unit array A sensitive pixel elements, each sensitive pixel elements include multiple photosensitive pixels, cover a photosensitive pixel The filter unit of unit be same color, the control method the following steps are included:
It controls described image sensor output and merges image, the merging image includes merging pixel array, same described photosensitive Multiple photosensitive pixels of pixel unit merge output and are used as a merging pixel；
Judge whether ambient brightness is less than or equal to preset threshold, the system information packet according to the system information of the electronic device Include system time；
When the ambient brightness is less than or equal to preset threshold, the merging image is converted to merging rgb image；
When the ambient brightness is greater than preset threshold, control described image sensor exports patch image, the patch image Image pixel elements including predetermined array arrangement, the predetermined array includes Bayer array, and described image pixel unit includes The original pixels of 2*2 array, each corresponding original pixels of the photosensitive pixel；
The patch image is converted to imitative original image, the imitative original image includes the imitative original pixel of array arrangement, the imitative original Pixel includes current pixel, and the original pixels include associated pixel corresponding with the current pixel, and the associated pixel exists Position in the patch image is corresponding with position of the current pixel in the imitative original image, described by the color lump figure As be converted to the imitative original image the step of the following steps are included:
Whether the color for judging the current pixel is identical as the color of the associated pixel；
When the color of the current pixel is identical as the color of the associated pixel, using the pixel value of the associated pixel as The pixel value of the current pixel；With
It is logical according to the pixel value of associated pixel unit in the color difference of the color and the associated pixel of the current pixel The pixel value that the first interpolation algorithm calculates the current pixel is crossed, described image pixel unit includes the associated pixel unit, The color of the associated pixel unit is identical as the current pixel and adjacent with the associated pixel；With
The imitative original image is converted to imitative former rgb image.
2. control method as described in claim 1, which is characterized in that described to be judged according to the system information of the electronic device The step of whether ambient brightness is less than or equal to preset threshold the following steps are included:
Judge whether the system time falls into predetermined amount of time；With
Determine that the ambient brightness is less than or equal to the preset threshold when the system time is fallen into the predetermined amount of time.
3. control method as claimed in claim 2, which is characterized in that the system information includes date information, the basis The system information of the electronic device judge the step of whether ambient brightness is less than or equal to preset threshold the following steps are included:
The predetermined amount of time is determined according to the date information.
4. control method as claimed in claim 2, which is characterized in that the system information includes geographical location information, described Judge the step of whether ambient brightness is less than or equal to preset threshold according to the system information of the electronic device the following steps are included:
The predetermined amount of time is determined according to the geographical location information.
5. control method as described in claim 1, which is characterized in that the electronic device includes GPS, the system information packet Include the signal strength of GPS.
6. control method as described in claim 1, which is characterized in that the electronic device for communicated with external server with Weather information is read, the system information includes the Weather information.
7. control method as described in claim 1, which is characterized in that the pixel value according to associated pixel unit passes through the One interpolation algorithm calculate the step of pixel value of the current pixel the following steps are included:
Calculate the gradual change amount in the associated pixel all directions；
Calculate the weight in the associated pixel all directions；With
According to the pixel value of current pixel described in the gradual change amount and the weight calculation.
8. control method as described in claim 1, which is characterized in that the control method is described according to associated pixel unit The pixel value the step of pixel value of the current pixel is calculated by the first interpolation algorithm before the following steps are included:
White balance compensation is done to the patch image；
The control method calculates the current picture by the first interpolation algorithm in the pixel value according to associated pixel unit Element pixel value the step of after the following steps are included:
White balance compensation reduction is done to the imitative original image.
9. control method as described in claim 1, which is characterized in that the control method is described according to associated pixel unit The pixel value the step of pixel value of the current pixel is calculated by the first interpolation algorithm before the following steps are included:
Compensating bad point is done to the patch image.
10. control method as described in claim 1, which is characterized in that the control method is described according to associated pixel list Before pixel value the step of calculating the pixel value of the current pixel by the first interpolation algorithm of member the following steps are included:
Crosstalk compensation is done to the patch image.
11. control method as described in claim 1, which is characterized in that the control method is described according to associated pixel list Include the following steps: after the step of pixel value of member calculates the pixel value of the current pixel by the first interpolation algorithm
Lens shape correction, demosaicing, noise reduction and edge sharpening processing are carried out to the imitative original image.
12. control method as described in claim 1, which is characterized in that described that the merging image is converted to merging very coloured silk The step of image the following steps are included:
Using the second interpolation algorithm by the merging image be converted to it is corresponding with the imitative original image go back original image, described second The complexity of interpolation algorithm is less than first interpolation algorithm；With
The original image of going back is converted to the merging rgb image.
13. a kind of control device, for controlling electronic device, which is characterized in that the electronic device includes imaging device, described Imaging device includes imaging sensor, and described image sensor includes sensitive pixel elements array and is arranged in the photosensitive pixel Filter unit array in cell array, the filter unit covering corresponding one of each of described filter unit array A sensitive pixel elements, each sensitive pixel elements include multiple photosensitive pixels, cover a photosensitive pixel The filter unit of unit is same color, and the control device includes:
First control module, first control module merge image, the merging for controlling described image sensor output Image includes merging pixel array, and multiple photosensitive pixels of the same sensitive pixel elements merge output and are used as a conjunction And pixel；
Judgment module, the judgment module are used to judge whether ambient brightness is less than according to the system information of the electronic device In preset threshold, the system information includes system time；
First conversion module, first conversion module are used for when the ambient brightness is less than or equal to preset threshold, will be described Merge image and is converted to merging rgb image；
Second control module, second control module are used to control the figure when the ambient brightness is greater than preset threshold As sensor output patch image, the patch image includes the image pixel elements of predetermined array arrangement, the predetermined array Including Bayer array, described image pixel unit includes the original pixels of 2*2 array, each corresponding institute of the photosensitive pixel State original pixels；
Second conversion module, second conversion module are used to the patch image being converted to imitative original image, the imitative original image Imitative original pixel as including array arrangement, the imitative original pixel includes current pixel, the original pixels include with it is described current The corresponding associated pixel of pixel, position of the associated pixel in the patch image and the current pixel are in the imitative original Position in image is corresponding, and second conversion module includes:
First judging unit, first judging unit is for judging the color of the current pixel and the face of the associated pixel Whether color is identical；
First computing unit, first computing unit are used for the color of color and the associated pixel in the current pixel When identical, using the pixel value of the associated pixel as the pixel value of the current pixel；With
Second computing unit, second computing unit are used for the color of color and the associated pixel in the current pixel When different, the pixel value of the current pixel is calculated by the first interpolation algorithm according to the pixel value of associated pixel unit, it is described Image pixel elements include the associated pixel unit, the color of the associated pixel unit it is identical as the current pixel and with The associated pixel is adjacent；With
Third conversion module, the third conversion module are used to for the imitative original image being converted to imitative former rgb image.
14. control device as claimed in claim 13, which is characterized in that the judgment module includes:
Second judgment unit, the second judgment unit is for judging whether the system time falls into predetermined amount of time；With
First determination unit, first determination unit are used to determine when the system time is fallen into the predetermined amount of time The ambient brightness is less than or equal to the preset threshold.
15. control device as claimed in claim 14, which is characterized in that the system information includes date information, described to sentence Disconnected module includes:
Second determination unit, second determination unit are used to determine the predetermined amount of time according to the date information.
16. control device as claimed in claim 14, which is characterized in that the system information includes geographical location information, institute Stating judgment module includes:
Third determination unit, the third determination unit are used to determine the predetermined amount of time according to the geographical location information.
17. control device as claimed in claim 13, which is characterized in that the electronic device includes GPS, the system information Signal strength including GPS.
18. control device as claimed in claim 13, which is characterized in that the electronic device with external server for communicating To read Weather information, the system information includes the Weather information.
19. control device as claimed in claim 13, which is characterized in that second computing unit includes:
First computation subunit, first computation subunit are used to calculate the gradual change amount in the associated pixel all directions；
Second computation subunit, second computation subunit are used to calculate the weight in the associated pixel all directions；With
Third computation subunit, the third computation subunit are used for current according to the gradual change amount and the weight calculation The pixel value of pixel.
20. control device as claimed in claim 13, which is characterized in that second conversion module includes:
White balance compensation unit, the white balance compensation unit is for doing white balance compensation to the patch image；
White balance compensation reduction unit, the white balance compensation reduction unit is for doing white balance compensation also to the imitative original image It is former.
21. control device as claimed in claim 13, which is characterized in that second conversion module includes:
Compensating bad point unit, the compensating bad point unit is for doing compensating bad point to the patch image.
22. control device as claimed in claim 13, which is characterized in that second conversion module includes:
Crosstalk compensation unit, the crosstalk compensation unit is for doing crosstalk compensation to the patch image.
23. control device as claimed in claim 13, which is characterized in that second conversion module includes:
Processing unit, the processing unit are used to carry out lens shape correction, demosaicing, noise reduction and side to the imitative original image Edge Edge contrast.
24. control device as claimed in claim 13, which is characterized in that first conversion module includes:
First conversion unit, first conversion unit are used to be converted to the merging image and institute using the second interpolation algorithm State that imitative original image is corresponding to go back original image, the complexity of second interpolation algorithm is less than first interpolation algorithm；With
Second conversion unit, second conversion unit are used to the original image of going back being converted to the merging rgb image.
25. a kind of electronic device, characterized by comprising:
Imaging device；With
Control device as described in claim 13-24 any one.
26. electronic device as claimed in claim 25, which is characterized in that the electronic device includes mobile phone or tablet computer.
27. electronic device as claimed in claim 25, which is characterized in that the imaging device includes Front camera or postposition phase Machine.
CN201611079187.4A 2016-11-29 2016-11-29 Control method, control device and electronic device Active CN106507018B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611079187.4A CN106507018B (en) 2016-11-29 2016-11-29 Control method, control device and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611079187.4A CN106507018B (en) 2016-11-29 2016-11-29 Control method, control device and electronic device

Publications (2)

Publication Number Publication Date
CN106507018A CN106507018A (en) 2017-03-15
CN106507018B true CN106507018B (en) 2019-05-10

Family

ID=58329160

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611079187.4A Active CN106507018B (en) 2016-11-29 2016-11-29 Control method, control device and electronic device

Country Status (1)

Country Link
CN (1) CN106507018B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106791477B (en) 2016-11-29 2019-07-19 Oppo广东移动通信有限公司 Image processing method, image processing apparatus, imaging device and manufacturing method
CN106504218B (en) 2016-11-29 2019-03-12 Oppo广东移动通信有限公司 Control method, control device and electronic device
CN106341670B (en) * 2016-11-29 2017-09-22 广东欧珀移动通信有限公司 Control method, control device and electronic installation
CN106454054B (en) 2016-11-29 2019-03-19 Oppo广东移动通信有限公司 Control method, control device and electronic device
CN107370917B (en) * 2017-06-30 2020-01-10 Oppo广东移动通信有限公司 Control method, electronic device, and computer-readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102857708A (en) * 2011-10-17 2013-01-02 北京瑞澜联合通信技术有限公司 Image sensor, photographing device and image data generation method
CN104168403A (en) * 2014-06-27 2014-11-26 深圳市大疆创新科技有限公司 High dynamic range video recording method and device based on Bayer color filtering array
CN105516696A (en) * 2015-12-18 2016-04-20 广东欧珀移动通信有限公司 Image sensor, imaging method, imaging device and electronic device
CN105578006A (en) * 2015-12-18 2016-05-11 广东欧珀移动通信有限公司 Imaging method, imaging device and electronic device
CN105611258A (en) * 2015-12-18 2016-05-25 广东欧珀移动通信有限公司 Imaging method of image sensor, imaging device and electronic device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102857708A (en) * 2011-10-17 2013-01-02 北京瑞澜联合通信技术有限公司 Image sensor, photographing device and image data generation method
CN104168403A (en) * 2014-06-27 2014-11-26 深圳市大疆创新科技有限公司 High dynamic range video recording method and device based on Bayer color filtering array
CN105516696A (en) * 2015-12-18 2016-04-20 广东欧珀移动通信有限公司 Image sensor, imaging method, imaging device and electronic device
CN105578006A (en) * 2015-12-18 2016-05-11 广东欧珀移动通信有限公司 Imaging method, imaging device and electronic device
CN105611258A (en) * 2015-12-18 2016-05-25 广东欧珀移动通信有限公司 Imaging method of image sensor, imaging device and electronic device

Also Published As

Publication number Publication date
CN106507018A (en) 2017-03-15

Similar Documents

Publication Publication Date Title
CN106504218B (en) Control method, control device and electronic device
CN106507018B (en) Control method, control device and electronic device
CN106454289B (en) Control method, control device and electronic installation
CN106412407B (en) Control method, control device and electronic device
CN106454054B (en) Control method, control device and electronic device
CN106341670B (en) Control method, control device and electronic installation
CN106791477B (en) Image processing method, image processing apparatus, imaging device and manufacturing method
CN106507067B (en) Control method, control device and electronic installation
CN106506984B (en) Image processing method and device, control method and device, imaging and electronic device
CN106507080B (en) Control method, control device and electronic device
CN106507069B (en) Control method, control device and electronic device
CN106604001B (en) Image processing method, image processing apparatus, imaging device and electronic device
CN106534822B (en) Control method, control device and electronic device
CN106412592B (en) Image processing method, image processing apparatus, imaging device and electronic device
CN106454288B (en) Control method, control device, imaging device and electronic installation
CN105578067B (en) image generating method, device and terminal device
CN106504217B (en) Image processing method, image processing apparatus, imaging device and electronic device
CN103685992A (en) Method for generating high dynamic range image and image sensor
CN107370917B (en) Control method, electronic device, and computer-readable storage medium
CN105611185A (en) Image generation method and device and terminal device
CN102508145A (en) Method for measuring CCD (Charge Coupled Device) chip dark current and double temperature constant
CN107507601A (en) A kind of method and system of the camera supervised interface display effect of adjust automatically
US10484622B2 (en) Method for improving image quality in images acquired by a near-infrared sensitive video camera and such a camera
JP2021097345A (en) Imaging apparatus, control method of the same, and program
CN117692807A (en) Visual sensor chip

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong

Applicant after: OPPO Guangdong Mobile Communications Co., Ltd.

Address before: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong

Applicant before: Guangdong OPPO Mobile Communications Co., Ltd.

GR01 Patent grant
GR01 Patent grant