CN106572289B - A kind of image processing method and mobile terminal of camera module - Google Patents
A kind of image processing method and mobile terminal of camera module Download PDFInfo
- Publication number
- CN106572289B CN106572289B CN201610921798.2A CN201610921798A CN106572289B CN 106572289 B CN106572289 B CN 106572289B CN 201610921798 A CN201610921798 A CN 201610921798A CN 106572289 B CN106572289 B CN 106572289B
- Authority
- CN
- China
- Prior art keywords
- image
- electric
- signal
- rgb
- electric signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Abstract
The embodiment of the invention provides a kind of image processing method of camera module and mobile terminals, the camera module are provided with imaging sensor, described image sensor has RGB output channel and IR output channel, which comprises acquisition incident ray；Described image sensor induction incident ray and conversion form electric image signal, include RGB image electric signal and IR electric image signal in described image electric signal；Described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；The RGB output channel exports the RGB image electric signal, and, the IR output channel exports the IR electric image signal；Target image is obtained according to the RGB image electric signal and the IR electric image signal.The embodiment of the present invention is to solve the problems, such as that dual camera excessively occupies mobile terminal space.
Description
Technical field
The present invention relates to fields of communication technology, image processing method and a kind of shifting more particularly to a kind of camera module
Dynamic terminal.
Background technique
With the continuous development of electronic products, the mobile terminal with shooting function is more more and more universal, and user can be at any time
It is shot everywhere with it.In order to obtain better shooting effect, be equipped on some mobile terminals on the market at present it is double after
Set camera.
Double rear cameras refer to that there are two camera modules for installation on the mobile terminal back side.Two camera modules can
To improve the shooting effect of mobile terminal.However, being executed since two camera modules in double rear cameras are independent mutually
It is independent of each other when shooting work, in other words, two camera modules respectively need the component of complete set, such as camera lens, figure
As sensor (sensor), microscope base, motor, substrate etc..It is appreciated that the camera module of this structure is on mobile terminals
The space occupied can be increased, and the certain functions of may result in mobile terminal are restricted, or damage mobile terminal
It is beautiful.
Summary of the invention
The embodiment of the present invention provides the image processing method and mobile terminal of a kind of camera module, to solve dual camera
Excessive the problem of occupying mobile terminal space.
In a first aspect, providing a kind of method of the image procossing of camera module, the method is applied to mobile terminal,
The camera module is provided with imaging sensor, described image sensor has RGB output channel and IR output channel, institute
The method of stating includes:
Acquire incident ray；
Described image sensor, which incudes incident ray and converts, forms electric image signal, includes in described image electric signal
RGB image electric signal and IR electric image signal；
Described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；
The RGB output channel exports the RGB image electric signal, and, the IR output channel exports the IR figure
As electric signal；
Target image is obtained according to the RGB image electric signal and the IR electric image signal.
Second aspect provides a kind of mobile terminal, and the mobile terminal includes camera module, in the camera mould
Group is provided with imaging sensor, and described image sensor has RGB output channel and IR output channel, the mobile terminal packet
It includes:
Incident ray acquisition module, for acquiring incident ray；
Electric image signal conversion module incudes incident ray for described image sensor and conversion forms image telecommunications
Number, it include RGB image electric signal and IR electric image signal in described image electric signal；
Electric image signal separation module isolates RGB image from described image electric signal for described image sensor
Electric signal and IR electric image signal；
Electric image signal output module exports the RGB image electric signal for the RGB output channel, and, it is described
IR output channel exports the IR electric image signal；
Target image generation module, for obtaining target according to the RGB image electric signal and the IR electric image signal
Image.
In this way, imaging sensor is arranged in the same camera module of mobile terminal in the embodiment of the present invention, image is passed
Sensor includes RGB output channel and IR output channel, can export RGB image electric signal and IR electric image signal respectively.When entering
When penetrating light and projecting camera module, electric image signal can be converted to based on image sensor senses incident ray, from this
Electric image signal isolates RGB image electric signal and IR electric image signal, exports RGB image telecommunications by corresponding output channel
Number and IR electric image signal, finally target image can be obtained based on rgb image data and IR image data.Using the present invention
Embodiment realizes the function of two camera modules in same camera module, can realize on the basis of hardware solely
Vertical output image data, does not need to handle by software.In addition, the embodiment of the present invention passes through a camera module
Realize that function possessed by dual camera mould group, this simple structure are not only effectively saved existing dual camera mould group and moved
The space occupied in dynamic terminal, and can also have the function of simultaneously possessed by dual camera mould group.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is a kind of step flow chart of the image processing method embodiment of camera module of the embodiment of the present invention one；
Fig. 2 is a kind of step flow chart of the image processing method embodiment of camera module of the embodiment of the present invention two；
Fig. 3 is a kind of structural block diagram of mobile terminal of the embodiment of the present invention three；
Fig. 3 a is a kind of one of the structural block diagram of mobile terminal of the embodiment of the present invention three；
Fig. 3 b is the two of the structural block diagram of a kind of mobile terminal of the embodiment of the present invention three；
Fig. 3 c is the three of the structural block diagram of a kind of mobile terminal of the embodiment of the present invention three；
Fig. 4 is a kind of block diagram of mobile terminal of the embodiment of the present invention four；
Fig. 5 is a kind of structural schematic diagram of mobile terminal of the embodiment of the present invention five.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Embodiment one
Fig. 1 is please referred to, is a kind of step flow chart of the image processing method of camera module of the embodiment of the present invention one.
A kind of image processing method of camera module provided in an embodiment of the present invention is applied to mobile terminal, described
Camera module is provided with imaging sensor, and described image sensor has RGB output channel and IR output channel, this method tool
Body may include steps of:
Step 101, incident ray is acquired.
It in the concrete realization, include camera lens in camera module, when light is reflected and is incident on from subject
When camera module, the incident ray of the camera lens acquisition subject in the camera module is used for subsequent processing operation.
Step 102, described image sensor, which incudes incident ray and converts, forms electric image signal, described image electric signal
In include RGB image electric signal and IR electric image signal.
In the camera module of the embodiment of the present invention, imaging sensor can be set to only one, can will incude
To incident ray be converted to the original image electric signal comprising RGB image electric signal and IR electric image signal.The image sensing
Device has at least two-way output channel, and one is RGB output channel, and another is IR output channel, the two channels can be divided
It Shu Chu not RGB image electric signal and IR electric image signal.
Imaging sensor, also referred to as photosensitive element are a kind of equipment for converting optical signals into electronic signal, are the present invention
One of the important component of camera module that embodiment provides.Imaging sensor provided in an embodiment of the present invention includes CCD
(Charge Coupled Device, charge coupled cell) and CMOS (Complementary Metal-Oxide
Semiconductor, metal oxide semiconductor device) two major classes.Specifically, above-mentioned CCD provides good picture quality, resists
Flexibility when ability of making an uproar and camera module design；But it needs to increase external circuit and makes becoming large-sized, being complicated for system
Property improve, needed in circuit design it is more flexible, to promote the performance of CCD camera mould group as far as possible.Above-mentioned CCD is more suitable for
It is such as astronomical, high-definition in the application field very high and less strict to cost control to camera module performance requirement
Medical X-ray image, other need time exposure and the scientific application stringent to picture noise requirement.
Above-mentioned CMOS is that have using contemporary large-scale semiconductive integrated circuit production technology come the imaging sensor produced
The features such as high yield rate, integrated level height, small power consumption, low price.Level of the CMOS in terms of image quality can be with CCD technology phase
Trial of strength, and they are more suitably applied to require space small, small in size, low in energy consumption and are not special to picture noise and quality requirement
High occasion.Such as most of industrial detection application for having filler lighting, security protection security personnel application and most of consumption-orientation business numbers
Code camera applications.
The imaging sensor of camera module mainly uses CMOS on mobile terminals at present, is situated between below for CMOS
It continues.
CMOS is a kind of typical solid state image sensor, usually by pixel array, line driver, row driver, timing
A few part compositions such as control logic, A/D converter, data/address bus output interface, control interface, this several part is usually all integrated
On same silicon wafer.
CMOS basic functional principle are as follows: firstly, extraneous incident ray irradiates pixel array, photoelectric effect occurs, in picture
Corresponding charge is generated in the pixel unit of pixel array to be converted to electric signal.The row selection logic unit of CMOS is according to need
It wants, gates corresponding row pixel unit.Electric image signal in row pixel unit is transmitted by the signal bus of respective column
To corresponding analogy signal processing unit and A/D converter, led to after being converted into digital picture electric signal by output accordingly
Road output.Row selection logic unit therein can progressively scan pixel array can also interlacing scan.Row selection logic unit
The window abstraction function that image may be implemented is used cooperatively with column selection logic unit.
Certainly, it can also be passed in embodiments of the present invention using other images in addition to CMOS and ccd image sensor
Sensor, as long as can be realized the acquisition of electric image signal, the embodiments of the present invention are not limited thereto.
Step 103, described image sensor isolates RGB image electric signal and IR image electricity from described image electric signal
Signal.
In the concrete realization, it can skip and ignore certain pixels on imaging sensor.The embodiment of the present invention is using such
Mode isolates required RGB image electric signal to can realize using an imaging sensor from electric image signal
With IR electric image signal, i.e., the separation of electric image signal can be realized on hardware, is not needed by way of software control
To realize.
Step 104, the RGB output channel exports the RGB image electric signal, and, the IR output channel output
The IR electric image signal.
In a particular application, imaging sensor may include have two output channels, so pass through an imaging sensor
RGB output channel and IR output channel, so that it may export respectively separated from electric image signal RGB image electric signal and
IR electric image signal.RGB image electric signal and IR electric image signal these data are saved in respectively in specified storage zone, can
To extract and use whenever necessary.
Step 105, target image is obtained according to the RGB image electric signal and the IR electric image signal.
It, can be according to reality for obtained RGB image electric signal and IR electric image signal in concrete application of the invention
The corresponding electric image signal of border demand selection individually carries out handling or synthesis is handled, such as passes through digital analog converter chip
It is converted to digital signal, to form required target image.Based on obtained target image, such as normal image bat may be implemented
It takes the photograph, with aware space and the depth of field, iris, makes the functions such as 3D rendering.
In this way, imaging sensor is arranged in the same camera module of mobile terminal in the embodiment of the present invention, image is passed
Sensor includes RGB output channel and IR output channel, can export RGB image electric signal and IR electric image signal respectively.When entering
When penetrating light and projecting camera module, electric image signal can be converted to based on image sensor senses incident ray, from this
Electric image signal isolates RGB image electric signal and IR electric image signal, exports RGB image telecommunications by corresponding output channel
Number and IR electric image signal, finally target image can be obtained based on rgb image data and IR image data.Using the present invention
Embodiment realizes the function of two camera modules in same camera module, can realize on the basis of hardware solely
Vertical output image data, does not need to handle by software.In addition, the embodiment of the present invention passes through a camera module
Realize that function possessed by dual camera mould group, this simple structure are not only effectively saved existing dual camera mould group and moved
The space occupied in dynamic terminal, and can also have the function of simultaneously possessed by dual camera mould group.
Embodiment two
Referring to figure 2., be the embodiment of the present invention two a kind of camera module image processing method step flow chart.
A kind of image processing method of camera module provided in an embodiment of the present invention is applied to mobile terminal, described
Camera module is provided with imaging sensor, and described image sensor has RGB output channel and IR output channel, this method tool
Body may include steps of:
Step 201, incident ray is acquired.
Step 202, described image sensor, which incudes incident ray and converts, forms electric image signal, described image electric signal
In include RGB image electric signal and IR electric image signal.
Since the step 201-202 of embodiment two is similar with embodiment of the method one, therefore the present embodiment is for step 201-202
Description in not detailed place, may refer to the related description in preceding method embodiment one, i will not repeat them here.
Step 203, described image sensor isolates RGB image electric signal and IR image electricity from described image electric signal
Signal.
In the concrete realization, imaging sensor, which can be realized, screens certain specific pixels, i.e., in imaging sensor
Can from electric image signal only screening RGB image electric signal, and, imaging sensor only filters out IR from electric image signal
Electric image signal, thus realize the separation of RGB image electric signal and IR electric image signal in electric image signal, such as:
In an embodiment of the present invention, IR electric image signal can be filtered out from image data, referring in particular to step
203 sub-steps as follows:
Described image sensor removes RGB image electric signal in described image electric signal line by line.
Described image sensor removes RGB image electric signal in described image electric signal by column.
By the electric image signal for having removed RGB image electric signal, as IR electric image signal.
It can be removed sampling with full line permutation in imaging sensor, in other words sample and arrange sampling individually in combination with row
Obtain IR electric image signal.Specifically, the row of no IR electric image signal can be first removed from electric image signal, then is removed
There is no the column of IR electric image signal, so that there is no RGB image electric signal in electric image signal, finally obtains IR electric image signal.
In an embodiment of the present invention, RGB image electric signal can be filtered out from image data, referring in particular to step
203 sub-steps as follows:
From the G component extracted in described image electric signal around the IR electric image signal；
Average G component is calculated based on the G component around described；
The IR electric image signal of the row is substituted using the average G component；
Using the electric image signal after the substitution as RGB image electric signal.
The pixel arrangement of the RGB-IR image data of figure as above, RGB data equally pass through removal full line without image of Buddha IR data
The mode of permutation obtains, but can increase image data device inside sensor, then utilizes image algorithm, IR pixel is replaced
It changes, the G of such as available surrounding pixel is averaged, and obtained G mends the IR pixel, finally obtains RGB image electric signal.
Step 204, the RGB output channel exports the RGB image electric signal, and, the IR output channel output
The IR electric image signal.
In embodiments of the present invention, the imaging sensor in camera module can be with only one, the image sensing utensil
There is multiple-channel output channel, can accordingly export RGB image electric signal and IR electric image signal respectively.
When imaging sensor generation electric image signal, and sampling obtains RGB image electric signal and IR from electric image signal
When electric image signal, electric image signal can be exported by corresponding output channel respectively.For example, being exported from RGB output channel
RGB image electric signal, and, IR electric image signal is exported by IR output channel.
Since imaging sensor has at least two output channels to export different image telecommunications in the same camera module
Number, that is, two-way output may be implemented in camera module hardware, it can while exporting RGB image electric signal and IR image electricity
Signal so that the function of completely self-contained two cameras may be implemented in same camera module, and is not required to carry out by software
Processing.
Step 205, colored target RGB image is obtained based on the RGB image electric signal, alternatively, scheming based on the IR
As electric signal obtains the target IR image of black and white, alternatively, being obtained based on the RGB image electric signal and the IR electric image signal
Target mixed image.
The camera module of the embodiment of the present invention can collect two-way electric image signal, i.e. RGB image electric signal and IR
Electric image signal is equivalent to there are two camera, has apparent shooting, shallow depth of field shooting and more preferably low light performance etc. excellent
Point.
Specifically, the first function of camera module of the embodiment of the present invention is that the shallow depth of field is supported to take pictures, and realizes that background is empty
Change, prominent photograph subject；Second function is that have better performance under low luminous environment.Camera mould through the embodiment of the present invention
Group can analyze which pixel to scene reduction degree it is relatively high, this can help mobile terminal effectively to inhibit noise, thus
So that image frame possesses more preferably image quality reduction.
For example, the embodiment of the present invention obtains conventional color image using RGB image electric signal, due to eliminating IR image
Electric signal, so that rubescent will not be distorted according to the image of RGB image electric signal；Black and white can also be obtained using IR electric image signal
Image, the characteristic based on infrared ray, resulting black white image can be with aware spaces and the depth of field；Further, acceptable while benefit
Mixed image is obtained with RGB image electric signal and IR electric image signal, in conjunction with the image of the two characteristic, can be and be able to reflect sky
Between information 3D rendering.
Certainly, RGB image electric signal and IR electric image signal are isolated for the camera module of the embodiment of the present invention, it can
To be applied according to actual needs, the embodiments of the present invention are not limited thereto.
In embodiments of the present invention, by an imaging sensor in the camera module of mobile terminal, on hardware
The separation and output that RGB image electric signal and IR electric image signal in electric image signal can be realized, do not need by software come
Processing, so that can realize the function of dual camera in a camera module.
Embodiment three
Referring to figure 3., be the embodiment of the present invention three a kind of mobile terminal 300 structural block diagram.
A kind of mobile terminal provided in an embodiment of the present invention, the mobile terminal may include camera module, described
Camera module is provided with imaging sensor, and described image sensor has RGB output channel and IR output channel, the movement
Terminal may include incident ray acquisition module 301, electric image signal conversion module 302, electric image signal separation module 303,
Electric image signal output module 304 and target image generation module 305.
Incident ray acquisition module 301, for acquiring incident ray.
Electric image signal conversion module 302, incudes incident ray for described image sensor and conversion forms image electricity
Signal includes RGB image electric signal and IR electric image signal in described image electric signal.
Electric image signal separation module 303 isolates RGB figure for described image sensor from described image electric signal
As electric signal and IR electric image signal.
Electric image signal output module 304 exports the RGB image electric signal for the RGB output channel, and,
The IR output channel exports the IR electric image signal.
Target image generation module 305 is obtained for root according to the RGB image electric signal and the IR electric image signal
Target image.
Referring to shown in Fig. 3 a, in a kind of mobile terminal 300 of the embodiment of the present invention, described image electric signal separation module
303 include the first removal submodule 3031, and second removes submodule 3032 and IR electric image signal acquisition submodule 3033.
First removes submodule 3031, removes RGB figure in described image electric signal line by line for described image sensor
As electric signal；
Second removes submodule 3032, removes RGB figure in described image electric signal by column for described image sensor
As electric signal；
IR electric image signal obtains submodule 3033, for by the electric image signal for having removed RGB image electric signal,
As IR electric image signal.
Referring to shown in Fig. 3 b, in a kind of mobile terminal 300 of the embodiment of the present invention, described image electric signal separation module
303 include G component extraction submodule 3034, and average G component computational submodule 3035IR electric image signal substitutes submodule 3036
Submodule 3037 is obtained with RGB image electric signal.
G component extraction submodule 3034, for from the G extracted in described image electric signal around the IR electric image signal
Component；
Average G component computational submodule 3035 calculates average G component for the G component based on the surrounding；
IR electric image signal substitutes submodule 3036, for substituting the IR image telecommunications of the row using the average G component
Number；
RGB image electric signal obtains submodule 3037, for using the electric image signal after the substitution as RGB image electricity
Signal.
The pixel arrangement of the RGB-IR image data of figure as above, RGB data equally pass through removal full line without image of Buddha IR data
The mode of permutation obtains, but can increase image data processor inside sensor, then utilize image algorithm, by IR pixel into
The G of row replacement, such as available surrounding pixel is averaged, and obtained G mends the IR pixel.Finally obtain RGB image electric signal.
Referring to shown in Fig. 3 c, in a kind of mobile terminal 300 of the embodiment of the present invention, the target image generation module 305
Submodule 3051 is obtained including target RGB image, target IR image obtains submodule 3052 and target mixed image obtains submodule
Block 3053.
Target RGB image obtains submodule 3051, for obtaining colored target RGB based on the RGB image electric signal
Image；
Target IR image obtains submodule 3052, and the target IR for being obtained black and white based on the IR electric image signal is schemed
Picture；
Target mixed image obtains submodule 3053, for being based on the RGB image electric signal and the IR image telecommunications
Number obtain target mixed image.
In a kind of mobile terminal 300 of the embodiment of the present invention, in of the imaging sensor of camera module setting
Number is one.
For 300 embodiment of mobile terminal, since it is basically similar to the method embodiment, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
In this way, imaging sensor is arranged in the same camera module of mobile terminal in the embodiment of the present invention, image is passed
Sensor includes RGB output channel and IR output channel, can export RGB image electric signal and IR electric image signal respectively.When entering
When penetrating light and projecting camera module, incident ray can be converted to by electric image signal based on imaging sensor, from the figure
As electric signal isolates RGB image electric signal and IR electric image signal, passes through corresponding output channel and export RGB image electric signal
With IR electric image signal, finally target image can be obtained based on rgb image data and IR image data.Using of the invention real
Example is applied, is equivalent in same camera module there are two camera module, independent output can be realized on the basis of hardware
Image data does not need to handle by software.In addition, the embodiment of the present invention can be realized by a camera module it is double
Function possessed by camera module, this simple structure not only effectively save first dual camera mould group on mobile terminals
The space occupied, and function possessed by dual camera mould group can also be retained simultaneously.
Example IV
Fig. 4 is the block diagram of the mobile terminal 700 of another embodiment of the present invention.
Mobile terminal 700 shown in Fig. 4 includes: at least one processor 701, memory 702, at least one network interface
704 and other users interface 703.Various components in mobile terminal 700 are coupled by bus system 705.It is understood that
Bus system 705 is for realizing the connection communication between these components.Bus system 705 is also wrapped in addition to including data/address bus
Include power bus, control bus and status signal bus in addition.But for the sake of clear explanation, various buses are all marked in Fig. 4
For bus system 705.
Wherein, user interface 703 may include display, keyboard or pointing device (for example, mouse, trace ball
(trackball), touch-sensitive plate or touch screen etc..
It is appreciated that the memory 702 in the embodiment of the present invention can be volatile memory or nonvolatile memory,
It or may include both volatile and non-volatile memories.Wherein, nonvolatile memory can be read-only memory (Read-
OnlyMemory, ROM), programmable read only memory (ProgrammableROM, PROM), Erasable Programmable Read Only Memory EPROM
(ErasablePROM, EPROM), electrically erasable programmable read-only memory (ElectricallyEPROM, EEPROM) dodge
It deposits.Volatile memory can be random access memory (RandomAccessMemory, RAM), and it is slow to be used as external high speed
It deposits.By exemplary but be not restricted explanation, the RAM of many forms is available, such as static random access memory
(StaticRAM, SRAM), dynamic random access memory (DynamicRAM, DRAM), Synchronous Dynamic Random Access Memory
(SynchronousDRAM, SDRAM), double data speed synchronous dynamic RAM (DoubleDataRate
SDRAM, DDRSDRAM), enhanced Synchronous Dynamic Random Access Memory (Enhanced SDRAM, ESDRAM), synchronized links
Dynamic random access memory (SynchlinkDRAM, SLDRAM) and direct rambus random access memory
(DirectRambusRAM, DRRAM).The memory 702 of the system and method for description of the embodiment of the present invention is intended to include but unlimited
In the memory of these and any other suitable type.
In some embodiments, memory 702 stores following element, executable modules or data structures, or
Their subset of person or their superset: operating system 7021 and application program 7022.
Wherein, operating system 7021 include various system programs, such as ccf layer, core library layer, driving layer etc., are used for
Realize various basic businesses and the hardware based task of processing.Application program 7022 includes various application programs, such as media
Player (MediaPlayer), browser (Browser) etc., for realizing various applied business.Realize embodiment of the present invention side
The program of method may be embodied in application program 7022.
In embodiments of the present invention, by the program or instruction of calling memory 702 to store, specifically, can be application
The program or instruction stored in program 7022, processor 701 is for acquiring incident ray；Described image sensor incudes incident light
Line and conversion form electric image signal, include RGB image electric signal and IR electric image signal in described image electric signal；The figure
As sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；The RGB output channel is defeated
The RGB image electric signal out, and, the IR output channel exports the IR electric image signal；According to the RGB image electricity
Signal and the IR electric image signal obtain target image.
The method that the embodiments of the present invention disclose can be applied in processor 701, or be realized by processor 701.
Processor 701 may be a kind of IC chip, the processing capacity with signal.During realization, the above method it is each
Step can be completed by the integrated logic circuit of the hardware in processor 701 or the instruction of software form.Above-mentioned processing
Device 701 can be general processor, digital signal processor (DigitalSignalProcessor, DSP), specific integrated circuit
(ApplicationSpecific IntegratedCircuit, ASIC), ready-made programmable gate array
(FieldProgrammableGateArray, FPGA) either other programmable logic device, discrete gate or transistor logic
Device, discrete hardware components.It may be implemented or execute disclosed each method, step and the logical box in the embodiment of the present invention
Figure.General processor can be microprocessor or the processor is also possible to any conventional processor etc..In conjunction with the present invention
The step of method disclosed in embodiment, can be embodied directly in hardware decoding processor and execute completion, or use decoding processor
In hardware and software module combination execute completion.Software module can be located at random access memory, and flash memory, read-only memory can
In the storage medium of this fields such as program read-only memory or electrically erasable programmable memory, register maturation.The storage
Medium is located at memory 702, and processor 701 reads the information in memory 702, and the step of the above method is completed in conjunction with its hardware
Suddenly.
It is understood that the embodiment of the present invention description these embodiments can with hardware, software, firmware, middleware,
Microcode or combinations thereof is realized.For hardware realization, processing unit be may be implemented in one or more specific integrated circuit (App
LicationSpecificIntegratedCircuits, ASIC), digital signal processor
(DigitalSignalProcessing, DSP), digital signal processing appts (DSPDevice, DSPD), programmable logic device
(ProgrammableLogicDevice, PLD), field programmable gate array (Field-ProgrammableGateArray,
FPGA), general processor, controller, microcontroller, microprocessor, other electronics lists for executing herein described function
In member or combinations thereof.
For software implementations, can by execute the embodiment of the present invention described in function module (such as process, function etc.) come
Realize technology described in the embodiment of the present invention.Software code is storable in memory and is executed by processor.Memory can
With portion realizes in the processor or outside the processor.
Optionally, processor 701 is also used to: described image sensor removes RGB figure in described image electric signal line by line
As electric signal；Described image sensor removes RGB image electric signal in described image electric signal by column；It has been removed described
The electric image signal of RGB image electric signal, as IR electric image signal.
Optionally, processor 701 is also used to: from the G extracted around the IR electric image signal in described image electric signal
Component；Average G component is calculated based on the G component around described；The IR image telecommunications of the row is substituted using the average G component
Number；Using the electric image signal after the substitution as RGB image electric signal.
Optionally, processor 701 is also used to: obtaining colored target RGB image based on the RGB image electric signal；Base
The target IR image of black and white is obtained in the IR electric image signal；And it is based on the RGB image electric signal and the IR image
Electric signal obtains target mixed image.
Mobile terminal 700 can be realized each process that mobile terminal is realized in previous embodiment, to avoid repeating, here
It repeats no more.
In this way, imaging sensor is arranged in the same camera module of mobile terminal in the embodiment of the present invention, image is passed
Sensor includes RGB output channel and IR output channel, can export RGB image electric signal and IR electric image signal respectively.When entering
When penetrating light and projecting camera module, electric image signal can be converted to based on image sensor senses incident ray, from this
Electric image signal isolates RGB image electric signal and IR electric image signal, exports RGB image telecommunications by corresponding output channel
Number and IR electric image signal, finally target image can be obtained based on rgb image data and IR image data.Using the present invention
Embodiment realizes the function of two camera modules in same camera module, can realize on the basis of hardware solely
Vertical output image data, does not need to handle by software.In addition, the embodiment of the present invention passes through a camera module
Realize that function possessed by dual camera mould group, this simple structure are not only effectively saved existing dual camera mould group and moved
The space occupied in dynamic terminal, and can also have the function of simultaneously possessed by dual camera mould group.
Embodiment five
Fig. 5 is the structural schematic diagram of the mobile terminal of another embodiment of the present invention.Specifically, the mobile terminal in Fig. 5
800 can be mobile phone, tablet computer, personal digital assistant (PersonalDigital Assistant, PDA) or vehicle-mounted computer
Deng.
Mobile terminal 800 in Fig. 5 includes radio frequency (RadioFrequency, RF) circuit 810, memory 820, input list
Member 830, display unit 840, processor 860, voicefrequency circuit 870, WiFi (WirelessFidelity) module 880 and power supply
890。
Wherein, input unit 830 can be used for receiving the number or character information of user's input, and generation and mobile terminal
The related signal input of 800 user setting and function control.Specifically, in the embodiment of the present invention, which can
To include touch panel 831.Touch panel 831, also referred to as touch screen collect the touch operation of user on it or nearby
(for example user uses the operations of any suitable object or attachment on touch panel 831 such as finger, stylus), and according to preparatory
The formula of setting drives corresponding attachment device.Optionally, touch panel 831 may include touch detecting apparatus and touch controller
Two parts.Wherein, the touch orientation of touch detecting apparatus detection user, and touch operation bring signal is detected, by signal
Send touch controller to；Touch controller receives touch information from touch detecting apparatus, and is converted into contact coordinate,
The processor 860 is given again, and can be received order that processor 860 is sent and be executed.Furthermore, it is possible to using resistance-type,
The multiple types such as condenser type, infrared ray and surface acoustic wave realize touch panel 831.In addition to touch panel 831, input unit
830 can also include other input equipments 832, other input equipments 832 can include but is not limited to physical keyboard, function key
One of (such as volume control button, switch key etc.), trace ball, mouse, operating stick etc. are a variety of.
Wherein, display unit 840 can be used for showing information input by user or be supplied to the information and movement of user
The various menu interfaces of terminal 800.Display unit 840 may include display panel 841, optionally, can use LCD or organic hair
The forms such as optical diode (OrganicLight-EmittingDiode, OLED) configure display panel 841.
It should be noted that touch panel 831 can cover display panel 841, touch display screen is formed, when the touch display screen is examined
After measuring touch operation on it or nearby, processor 860 is sent to determine the type of touch event, is followed by subsequent processing device
860 provide corresponding visual output according to the type of touch event in touch display screen.
Touch display screen includes Application Program Interface viewing area and common control viewing area.The Application Program Interface viewing area
And arrangement mode of the common control viewing area does not limit, can be arranged above and below, left-right situs etc. can distinguish two it is aobvious
Show the arrangement mode in area.The Application Program Interface viewing area is displayed for the interface of application program.Each interface can be with
The interface elements such as the icon comprising at least one application program and/or widget desktop control.The Application Program Interface viewing area
Or the empty interface not comprising any content.This commonly uses control viewing area for showing the higher control of utilization rate, for example,
Application icons such as button, interface number, scroll bar, phone directory icon etc. are set.
Wherein processor 860 is the control centre of mobile terminal 800, utilizes various interfaces and connection whole mobile phone
Various pieces, by running or executing the software program and/or module that are stored in first memory 821, and calling storage
Data in second memory 822 execute the various functions and processing data of mobile terminal 800, thus to mobile terminal 800
Carry out integral monitoring.Optionally, processor 860 may include one or more processing units.
In embodiments of the present invention, by call store the first memory 821 in software program and/or module and/
Or the data in the second memory 822, processor 860 is for acquiring incident ray；Described image sensor incudes incident light
Line and conversion form electric image signal, include RGB image electric signal and IR electric image signal in described image electric signal；The figure
As sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；The RGB output channel is defeated
The RGB image electric signal out, and, the IR output channel exports the IR electric image signal；According to the RGB image electricity
Signal and the IR electric image signal obtain target image.
The processor 860 is also used to: described image sensor removes RGB image electricity in described image electric signal line by line
Signal；Described image sensor removes RGB image electric signal in described image electric signal by column；RGB figure has been removed by described
As the electric image signal of electric signal, as IR electric image signal.
The processor 860 is also used to: from the G extracted around the IR electric image signal in described image electric signal points
Amount；Average G component is calculated based on the G component around described；The IR image telecommunications of the row is substituted using the average G component
Number；Using the electric image signal after the substitution as RGB image electric signal.
The processor 860 is also used to: obtaining colored target RGB image based on the RGB image electric signal；Based on institute
It states IR electric image signal and obtains the target IR image of black and white；And it is based on the RGB image electric signal and the IR image telecommunications
Number obtain target mixed image.
For 800 embodiment of mobile terminal, since it is basically similar to the method embodiment, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
In this way, imaging sensor is arranged in the same camera module of mobile terminal in the embodiment of the present invention, image is passed
Sensor includes RGB output channel and IR output channel, can export RGB image electric signal and IR electric image signal respectively.When entering
When penetrating light and projecting camera module, electric image signal can be converted to based on image sensor senses incident ray, from this
Electric image signal isolates RGB image electric signal and IR electric image signal, exports RGB image telecommunications by corresponding output channel
Number and IR electric image signal, finally target image can be obtained based on rgb image data and IR image data.Using the present invention
Embodiment realizes the function of two camera modules in same camera module, can realize on the basis of hardware solely
Vertical output image data, does not need to handle by software.In addition, the embodiment of the present invention passes through a camera module
Realize that function possessed by dual camera mould group, this simple structure are not only effectively saved existing dual camera mould group and moved
The space occupied in dynamic terminal, and can also have the function of simultaneously possessed by dual camera mould group.
Those of ordinary skill in the art may be aware that the embodiment in conjunction with disclosed in the embodiment of the present invention describe it is each
Exemplary unit and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond the scope of this invention.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In embodiment provided herein, it should be understood that disclosed device and method can pass through others
Mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of device or unit
It connects, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, ROM, RAM, magnetic or disk etc. are various can store program code
Medium.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be subject to the protection scope in claims.
Claims (7)
1. a kind of image processing method of camera module is applied to mobile terminal, which is characterized in that in the camera module
It is provided with imaging sensor, described image sensor has RGB output channel and IR output channel, which comprises
Acquire incident ray；
Described image sensor induction incident ray and conversion form electric image signal, include that RGB schemes in described image electric signal
As electric signal and IR electric image signal；
Described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal；
The RGB output channel exports the RGB image electric signal, and, the IR output channel exports the IR image electricity
Signal；
Target image is obtained according to the RGB image electric signal and the IR electric image signal；
Wherein, described image sensor isolates RGB image electric signal and IR electric image signal from described image electric signal
Step includes:
From the G component extracted in described image electric signal around the IR electric image signal；
Average G component is calculated based on the G component around described；
The IR electric image signal is substituted using the average G component；
Using the electric image signal after the substitution as RGB image electric signal.
2. the method according to claim 1, wherein described according to RGB image electric signal and IR image electricity
Signal obtains the step of target image and includes:
Colored target RGB image is obtained based on the RGB image electric signal；
The target IR image of black and white is obtained based on the IR electric image signal；
And target mixed image is obtained based on the RGB image electric signal and the IR electric image signal.
3. the method according to claim 1, wherein of the imaging sensor in camera module setting
Number is one.
4. a kind of mobile terminal, which is characterized in that the mobile terminal includes camera module, is arranged in the camera module
There is imaging sensor, described image sensor has RGB output channel and IR output channel, and the mobile terminal includes:
Incident ray acquisition module, for acquiring incident ray；
Electric image signal conversion module incudes incident ray for described image sensor and conversion forms electric image signal, institute
State includes RGB image electric signal and IR electric image signal in electric image signal；
Electric image signal separation module isolates RGB image telecommunications for described image sensor from described image electric signal
Number and IR electric image signal；
Electric image signal output module exports the RGB image electric signal for the RGB output channel, and, the IR is defeated
Access exports the IR electric image signal out；
Target image generation module, for obtaining target image according to the RGB image electric signal and the IR electric image signal；
Wherein, described image electric signal separation module includes:
G component extraction submodule, for from the G component extracted in described image electric signal around the IR electric image signal；
Average G component computational submodule calculates average G component for the G component based on the surrounding；
IR electric image signal substitutes submodule, for substituting the IR electric image signal using the average G component；
RGB image electric signal obtains submodule, for using the electric image signal after the substitution as RGB image electric signal.
5. mobile terminal according to claim 4, which is characterized in that described image electric signal separation module includes:
First removes submodule, removes RGB image electric signal in described image electric signal line by line for described image sensor；
Second removes submodule, removes RGB image electric signal in described image electric signal by column for described image sensor；
IR electric image signal obtains submodule, for being schemed as IR by the electric image signal for having removed RGB image electric signal
As electric signal.
6. mobile terminal according to claim 4, which is characterized in that the target image generation module includes:
Target RGB image obtains submodule, for obtaining colored target RGB image based on the RGB image electric signal；
Target IR image obtains submodule, for obtaining the target IR image of black and white based on the IR electric image signal；
Target mixed image obtains submodule, for obtaining mesh based on the RGB image electric signal and the IR electric image signal
Mark mixed image.
7. mobile terminal according to claim 4, which is characterized in that in the imaging sensor of camera module setting
Number be one.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610921798.2A CN106572289B (en) | 2016-10-21 | 2016-10-21 | A kind of image processing method and mobile terminal of camera module |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610921798.2A CN106572289B (en) | 2016-10-21 | 2016-10-21 | A kind of image processing method and mobile terminal of camera module |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106572289A CN106572289A (en) | 2017-04-19 |
| CN106572289B true CN106572289B (en) | 2019-08-20 |
Family
ID=60414372
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610921798.2A Active CN106572289B (en) | 2016-10-21 | 2016-10-21 | A kind of image processing method and mobile terminal of camera module |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106572289B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109951646B (en) * | 2017-12-20 | 2021-01-15 | 杭州海康威视数字技术股份有限公司 | Image fusion method and device, electronic equipment and computer readable storage medium |
| CN108965654B (en) | 2018-02-11 | 2020-12-25 | 浙江宇视科技有限公司 | Double-spectrum camera system based on single sensor and image processing method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103376617A (en) * | 2012-04-16 | 2013-10-30 | 索尼公司 | Image sensor, control method of image sensor, and imaging apparatus |
| CN104463112A (en) * | 2014-11-27 | 2015-03-25 | 深圳市科葩信息技术有限公司 | Method and system for carrying out biological recognition through RGB+IR image sensor |
| CN105763848A (en) * | 2016-03-03 | 2016-07-13 | 浙江宇视科技有限公司 | Fish-eye camera rear end access method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9996726B2 (en) * | 2013-08-02 | 2018-06-12 | Qualcomm Incorporated | Feature identification using an RGB-NIR camera pair |
-
2016
- 2016-10-21 CN CN201610921798.2A patent/CN106572289B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103376617A (en) * | 2012-04-16 | 2013-10-30 | 索尼公司 | Image sensor, control method of image sensor, and imaging apparatus |
| CN104463112A (en) * | 2014-11-27 | 2015-03-25 | 深圳市科葩信息技术有限公司 | Method and system for carrying out biological recognition through RGB+IR image sensor |
| CN105763848A (en) * | 2016-03-03 | 2016-07-13 | 浙江宇视科技有限公司 | Fish-eye camera rear end access method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106572289A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106657726B (en) | A kind of image processing method and mobile terminal of camera module | |
| CN105827754B (en) | A kind of generation method and mobile terminal of high dynamic range images | |
| CN107205120B (en) | A kind of processing method and mobile terminal of image | |
| CN105827964B (en) | A kind of image processing method and mobile terminal | |
| CN106101545B (en) | A kind of image processing method and mobile terminal | |
| CN107222680A (en) | The image pickup method and mobile terminal of a kind of panoramic picture | |
| CN107231530B (en) | A kind of photographic method and mobile terminal | |
| CN106331510A (en) | Backlight photographing method and mobile terminal | |
| CN105474626B (en) | Photographic device | |
| CN107277387A (en) | High dynamic range images image pickup method, terminal and computer-readable recording medium | |
| CN106170058B (en) | A kind of exposure method and mobile terminal | |
| CN106101544B (en) | A kind of image processing method and mobile terminal | |
| CN105827970A (en) | Image processing method and mobile terminal | |
| CN107465874B (en) | A kind of dark current processing method and mobile terminal | |
| CN106161943A (en) | A kind of kinescope method and mobile terminal | |
| CN106454085B (en) | A kind of image processing method and mobile terminal | |
| CN106973233A (en) | A kind of camera, mobile terminal and processing method of taking pictures | |
| CN105979156A (en) | Panoramically photographing method and mobile terminal | |
| CN106572290B (en) | A kind of processing method and mobile terminal of image data | |
| CN106572289B (en) | A kind of image processing method and mobile terminal of camera module | |
| CN106454086A (en) | Image processing method and mobile terminal | |
| CN110113519A (en) | Camera module | |
| CN107613164A (en) | A kind of camera module and mobile terminal with fingerprint identification function | |
| CN108370417A (en) | The mobile terminal and screening-mode conversion method of screening-mode can be converted easily | |
| CN106453886B (en) | A kind of photographic method and mobile terminal of mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |