CN107493411B - Image processing system and method - Google Patents
Image processing system and method Download PDFInfo
- Publication number
- CN107493411B CN107493411B CN201710676485.XA CN201710676485A CN107493411B CN 107493411 B CN107493411 B CN 107493411B CN 201710676485 A CN201710676485 A CN 201710676485A CN 107493411 B CN107493411 B CN 107493411B
- Authority
- CN
- China
- Prior art keywords
- image
- light
- depth
- structure light
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of image processing system and methods, wherein system includes: structured light projector, for projecting default projection pattern structure light to shooting main body；First camera, for shooting first structure light image of the default projection pattern structure light by shooting main body modulation, second camera, for shooting second structure light image of the default projection pattern structure light by shooting main body modulation；Image-signal processor, for being demodulated to first structure light image to extract depth value greater than prospect threshold value and be less than the first depth of view information of background threshold value, second structure light image is demodulated to extract depth value greater than prospect threshold value and be less than the second depth of view information of background threshold value, and image co-registration is carried out to generate target image according to the first depth of view information and the second depth of view information.The obtained target image of the system is more truly reflected shooting main body, and shooting effect is good, user experience is good.
Description
Technical field
The present invention relates to technical field of information processing more particularly to a kind of image processing system and methods.
Background technique
As the camera function of the mobile terminals such as mobile phone, plate is stronger and stronger, taken pictures using cell phone replacement camera
People it is also more and more, configuring high performance image processing system on mobile terminals and being able to ascend to take pictures effect and ensures good
User experience.
There is the image processing system one is dual camera in current image processing system, such as one in dual camera
For focal length camera, one carries out selecting a selection in dual camera for wide-angle camera according to the difference of photographed scene.
However, the performance of taking pictures of the image processing system of existing dual camera is equivalent to and is configured with two on mobile terminals
The performance of taking pictures of the image processing system of a different single camera, the image processing system of dual camera can not be than single camera shooting
How much superior the image processing system of head is.Therefore, the performance for how improving the image processing system of dual camera, which becomes, urgently to be solved
Certainly the technical issues of.
Summary of the invention
The present invention provides a kind of image processing system and method, to solve in the prior art, the user of image processing system
Experience the problem of difference.
First aspect of the embodiment of the present invention provides a kind of image processing system, comprising: structured light projector, the first camera shooting
Head, second camera, image-signal processor；
The structured light projector with it is described first image altogether cross central line be arranged successively, and with the second camera
Longitudinal centre line is arranged successively altogether；
The structured light projector, for projecting default projection pattern structure light to shooting main body；
First camera, for shooting the default projection pattern structure light the by the shooting main body modulation
One structure light image,
The second camera, for shooting the default projection pattern structure light the by the shooting main body modulation
Two structure light images；
Described image signal processor, for being demodulated to the first structure light image to extract before depth value is greater than
Scape threshold value and the first depth of view information for being less than background threshold value, demodulate second structure light image big to extract depth value
In prospect threshold value and less than the second depth of view information of background threshold value, and according to first depth of view information and second depth of field
Information carries out image co-registration to generate target image.
Second aspect of the embodiment of the present invention provides a kind of image processing method, comprising: structured light projector, the first camera shooting
Head, second camera, image-signal processor；
The structured light projector with it is described first image altogether cross central line be arranged successively, and with the second camera
Longitudinal centre line is arranged successively altogether；
The structured light projector projects default projection pattern structure light to shooting main body；
First camera shoots first knot of the default projection pattern structure light by the shooting main body modulation
Structure light image,
The second camera shoots second knot of the default projection pattern structure light by the shooting main body modulation
Structure light image；
Described image signal processor demodulates the first structure light image to extract depth value greater than prospect threshold
Value and the first depth of view information for being less than background threshold value demodulate second structure light image to extract before depth value is greater than
Scape threshold value and the second depth of view information for being less than background threshold value, and according to first depth of view information and second depth of view information
Image co-registration is carried out to generate target image.
The third aspect of the embodiment of the present invention provides a kind of terminal device, including memory and processor, in the memory
Computer-readable instruction is stored, when described instruction is executed by the processor, so that the processor executes the present invention second
Image processing method described in aspect embodiment.
Fourth aspect of the embodiment of the present invention provides a kind of non-transitorycomputer readable storage medium, is stored thereon with calculating
Machine program realizes the image processing method as described in second aspect of the present invention embodiment when the computer program is executed by processor
Method.
Technical solution provided in an embodiment of the present invention can include the following benefits:
Firstly, structured light projector projects default projection pattern structure light to shooting main body；Then, the first camera is shot
First structure light image of the default projection pattern structure light by shooting main body modulation, second camera shoot default projection pattern
Second structure light image of the structure light by shooting main body modulation；Image-signal processor demodulates first structure light image
To extract depth value greater than prospect threshold value and be less than the first depth of view information of background threshold value, the second structure light image is demodulated
To extract depth value greater than prospect threshold value and be less than the second depth of view information of background threshold value, and according to the first depth of view information and the
Two depth of view information carry out image co-registration to generate target image.Target image obtained is more truly reflected shooting main body,
Shooting effect is good, user experience is good.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, in which:
Fig. 1 is the structural schematic diagram of the image processing system of one embodiment of the invention；
Fig. 2 be structured light projector in Fig. 1, the first camera, second camera arrangement schematic diagram；
Fig. 3 is the flow chart of the image processing method of one embodiment of the invention；
Fig. 4 is the structural schematic diagram of the image processing circuit in terminal device according to an embodiment of the invention.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to is used to explain the present invention, and is not considered as limiting the invention.
Below with reference to the accompanying drawings the image processing system and method for the embodiment of the present invention are described.Wherein, the image processing system
It is desirably integrated into mobile terminal.Before introducing the image processing system of the embodiment of the present invention, briefly it is situated between to structured light technique
It continues.Structured light technique is exactly pattern (such as discrete light spot, fringe light, the coding using designed in advance with special construction
Structure light etc.), it then projects a pattern on three-dimensional object surface, using imaging device such as camera observation in three-dimensional article
Manage the distortion situation of surface imaging.If it is a plane that structured light patterns, which are projected in the body surface, observe at
The pattern of structure light is just similar with the pattern of projection as in, does not deform, and only generates certain scale according to distance and becomes
Change.But if body surface is not plane, geometry that the structured light patterns observed will be different because of body surface
Shape and generate different torsional deformations, and it is different according to the difference of distance, according to known structured light patterns and observation
The deformation arrived can calculate the 3D shape and depth information of measured object according to algorithm.That is, working as in the real world
When object projects the structure light of certain projection pattern, the structure light of certain projection pattern reflects on the surface of object, by
All be in object in the real world it is three-dimensional, being reflected by the object structure light back no longer has phase with the structure light before reflection
Same pattern may be implemented by comparing the projection pattern of the structure light before reflecting and the deformation pattern of the structure light after reflection
The three-dimensional spatial information for fast and accurately obtaining object in real world, structured light technique is applied in photographed scene, is shot
Image good imaging quality, the precision arrived is high.
Fig. 1 is the structural schematic diagram of the image processing system of one embodiment of the invention.As shown in Figure 1, the present embodiment provides
Image processing system, comprising: structured light projector 10, the first camera 20, second camera 30, image-signal processor
1130。
Fig. 2 be structured light projector 10 in Fig. 1, the first camera 20, second camera 30 arrangement schematic diagram.Referring to
Fig. 2, the structured light projector 10 in the present embodiment are total to cross central line with the first camera 20 and are arranged successively, and with the second camera shooting
First 30 total longitudinal centre lines are arranged successively.For example, structured light projector 10 and the first camera 20 be total to cross central line according to
Secondary arrangement is structured light projector 10 and the first camera 20 in the horizontal direction at a distance of certain spacing arrangement, structured light projector 10
Longitudinal centre line, which is total to, with second camera 30 is arranged successively i.e. structured light projector 10 and the first camera 20 in longitudinal direction phase
Away from certain spacing arrangement, such structured light projector 10, the first camera 20, second camera 30 can be in the form of arranged at right angles
It is arranged in the terminal, since the mutual spacing of structured light projector 10, the first camera 20, second camera 30 is straight
Influence image quality is connect, therefore, the mass parameter of taking pictures of mobile terminal is configured in strict accordance with the manufacturer of mobile terminal
The mutual spacing of structured light projector 10, the first camera 20, second camera 30.
Specifically, structured light projector 10, for projecting default projection pattern structure light to shooting main body.For example,
Projection pattern in the present embodiment can be laser stripe, Gray code, sine streak or, the speckle pattern etc. of random alignment.
Structured light projector 10 in the present embodiment can project projection pattern structure light of various shapes.For example, one is utilized
A light source controller, an array light source can design the structured light projector 10 in the present embodiment, and light source controller can
According to the luminance of each light source in selected projection pattern control array light source, the light that all light sources being lit issue
As projection pattern structure light.
In one possible implementation, structured light projector 10 includes light source controller, array light source；Light source control
Device, for the luminance according to each light source for presetting projection pattern control array light source, luminance includes: illuminating state
And OFF state；Array light source presets projection pattern knot for issuing default projection pattern project structured light to shooting main body
Structure light is the structure light that the light that the light source of whole illuminating states issues is formed.
Specifically, object in the real world is multifarious, and the surface topography of some objects is very uneven, as long as at this moment
The structure light fairly simple to project objects pattern also has bigger deflection by the structure light of object modulation back, just
It may insure the image quality of target image；The surface topography of some objects is very smooth, at this moment needs to project objects pattern ratio
More complex structure light, with guarantee by object modulation back structure light have bigger deflection, it is ensured that target image at
Image quality amount.Different projection pattern structure lights of various shapes are formed compared to the grating using different shapes, in the present embodiment
Projection pattern structure light of various shapes is formed using light source controller, array light source to be simple and efficient, control simple, can satisfy
Shooting demand under different scenes.Preferably, array light source is LED (Light-Emitting Diode, light emitting diode) battle array
Column light source.LED (Light-Emitting Diode, light emitting diode) array light source have low-power consumption, high brightness, it is low in calories,
The advantages such as small in size, long service life, environmental protection, voltage is low.When by LED (Light-Emitting Diode, light emitting diode)
When array light source is applied in the structured light projector 10 in the present embodiment, the structured light projector for the miniaturization that can be produced
10, correspondingly, when structured light projector 10 is integrated into mobile terminal, too many space will not be occupied, mobile terminal is met
More lighter, thinner development trend.
Further, in this embodiment image processing system further include: light sensor, for obtaining external environmental light
Light intensity；Correspondingly, light source controller is used to control the luminous intensity of the light source of illuminating state according to light intensity.Citing
For, it may determine that current environment is daytime or night by the detection of light sensor, if display current environment is daytime,
To guarantee that overwhelming majority is the structure light modulated by shooting main body, at this moment control structure light projection in the received light of camera
The light intensity that device 10 issues is greater than the light intensity of external environmental light, to guarantee good shooting effect.Obviously, if display is worked as
Preceding environment is night, and at this moment structured light projector 10 has both the effect of light compensating lamp, and it is good to guarantee to issue the structure light of sufficient intensity
Good shooting effect.The present embodiment controls the luminous intensity of the light source of illuminating state by light source controller according to light intensity,
And then realize the intensity for the structure light that structured light projector 10 issues, it controls simple and convenient.
Wherein, the first camera 20, for shooting first knot of the default projection pattern structure light by shooting main body modulation
Structure light image；Second camera 30, for shooting second structure light of the default projection pattern structure light by shooting main body modulation
Image.
Specifically, since the first camera 20, second camera 30 are arranged at the different location of structured light projector 10,
By the mutual spacing of the first camera 20 of control, second camera 30, structured light projector 10, may be implemented entirely to clap
It takes the photograph main body and is all predetermined the irradiation of projection pattern structure light, it is subsequent again by first structure light image, second structure light image
Image procossing, it will be able to realize that obtained target image is more truly reflected shooting main body, be conducive to improve shooting effect and
User experience.
Specifically, the type of the camera in the present embodiment is unlimited, can be wide-angle camera, is also possible to focal length camera shooting
Head.The basic image-forming principle of camera are as follows: the imaging sensor built in camera receives the light reflected by real-world objects,
Since imaging sensor is optical sensor, the received light being reflected by the object can be converted into the electric signal of characterization object,
Subject image is exported after carrying out information processing to the electric signal of characterization object.Therefore, the first structure light image in the present embodiment
Forming process substantially are as follows: project default projection pattern structure light to shooting main body using structured light projector 10, preset projection
Reflection (i.e. default projection pattern structure light is by the modulation of shooting main body), camera benefit occur in shooting main body for patterning light
The reflected structure light of main body that is taken is received with built-in imaging sensor, first structure light is exported after information processing
Image.The forming process of the second structure light image in the present embodiment is substantially are as follows: utilizes structured light projector 10 to shooting main body
Default projection pattern structure light is projected, projection pattern structure light is preset and reflection (i.e. default projection pattern occurs in shooting main body
Structure light is by the modulation of shooting main body), camera receives the reflected knot of main body that is taken using built-in imaging sensor
Structure light exports the second structure light image after information processing.Preferably, the first camera 20 is clapped simultaneously with second camera 30
It takes the photograph.
Wherein, image-signal processor 1130, for being demodulated to first structure light image to extract depth value and be greater than
Prospect threshold value and the first depth of view information for being less than background threshold value, demodulate the second structure light image to extract depth value and be greater than
Prospect threshold value and the second depth of view information for being less than background threshold value, and figure is carried out according to the first depth of view information and the second depth of view information
As fusion is to generate target image.
Specifically, by being demodulated to structure light image, by calculating the deformation degree of projection pattern and utilizing triangle
The depth value of each point of the available shooting main body of geometrical principle.The depth of each point in photographed scene is obtained using structured light technique
Value is not repeating herein referring to the prior art.
Prospect refers in shooting main body outrunner object or scenery play the role of setting off main body by contrast or directly help to express theme,
And the spatial depth of picture can be enhanced, balanced and beautification picture.Prospect threshold value in the present embodiment can be according to according to mobile whole
The manufacturer at end is configured the mass parameter of taking pictures of mobile terminal, and multiple, different prospect thresholds can be set in prospect threshold value
It is worth corresponding shooting effect, can satisfy the shooting demand of different user.For example, user has selected a prospect threshold value, that
The shooting picture that depth value is greater than prospect threshold value, which can all appear in, to be presented in final target image, and final mesh is presented on
Logo image has: shooting main body and personage of the depth value greater than prospect threshold value or scenery before shooting main body.
Background refers to close to the subsequent personage of shooting main body or scenery, so that picture is generated the modeling effect of multilayer scenery, increases
Strong spatial depth sense.Background threshold value in the present embodiment can take pictures to mobile terminal according to according to the manufacturer of mobile terminal
Mass parameter is configured, and background threshold value can be set multiple, different background threshold values and correspond to shooting effect, can satisfy difference
The shooting demand of user.For example, user has selected a background threshold value, then depth value is less than the shooting picture of background threshold value
Face can all appear in and be presented in final target image, and being presented in final target image has: shooting main body and clapping
Take the photograph personage or scenery that depth value behind main body is less than background threshold value.
Specifically, depth of view information, which refers to, can obtain the imaging of clear image in camera lens or other imager forward positions
The subject longitudinal separation range measured.In brief, it after the completion of focusing, can be formed in the range of before and after focus
Clearly as this distance range one in front and one in back is depth of view information.Under normal circumstances, the depth of field is bigger, between the scape of front and back
Distance is remoter, and the depth feelings of picture are stronger, more can increase the multi-level and associative perception of living environment in film.In the present embodiment
In, the first depth of view information or the corresponding image of the second depth of view information include: shooting main body, the depth value before shooting main body
Depth value is less than personage or the scenery of background threshold value greater than the personage of prospect threshold value or scenery and behind shooting main body.This reality
It applies example and passes through depth value compared with prospect threshold value, background threshold value, obtain the first depth of view information and the second depth of view information to choose
Clearly image in current shooting picture, in this way, subsequent be based on the first depth of view information and the obtained mesh of the second depth of view information
Logo image good imaging quality, shooting effect are good.Further, it is also possible to which multiple prospect threshold values, multiple background threshold values are arranged, target is controlled
Prospect and background in image, mise-en-scene when convenient for shooting, can satisfy the shooting demand of different user.
Except can ensure that the clear of final target image by controlling acquired the first depth of view information and the second depth of view information
Except clear degree, it is also necessary to guarantee that the degree of overlapping for generating target image according to first structure light image and the second structure light image is high.
Further, image-signal processor 1130 is specifically used for: determining image weight according to the first depth of view information and the first depth of view information
Folded area's information；Image segmentation is carried out to first structure light image according to image overlay region information, obtains the first overlapping image and the
One non-overlapping images；Image segmentation is carried out to the second structure light image according to image overlay region information, obtains the second overlapping image
With the second non-overlapping images；It chooses the first overlapping image and the second overlapping image carries out image co-registration to generate target image.It needs
It is noted that image co-registration is pith in image procossing, the multiple sensors image using Same Scene can be cooperateed with
Information, one width of output are further processed the blending image with analysis more suitable for human visual perception or computer.It can be obvious
Improvement single-sensor deficiency, improve the clarity and packet content of image, be conducive to it is more accurate, more reliable,
More fully obtain the information of target or scene.After the previously determined first overlapping image of the present embodiment and the second overlapping image
The target image for carrying out image co-registration generation again has better clarity and degree of overlapping, and can eliminate as much as possible in image
The dislocation of fusion process occurs.
Image processing system provided in an embodiment of the present invention, firstly, structured light projector 10 is default to shooting main body projection
Projection pattern structure light；Then, the first camera 20 shoots first of default projection pattern structure light by shooting main body modulation
Structure light image, second camera 30 shoot second structure light figure of the default projection pattern structure light by shooting main body modulation
Picture；Image-signal processor 1130 demodulates first structure light image to extract depth value greater than prospect threshold value and after being less than
First depth of view information of scape threshold value demodulates the second structure light image to extract depth value greater than prospect threshold value and after being less than
Second depth of view information of scape threshold value, and image co-registration is carried out to generate target according to the first depth of view information and the second depth of view information
Image.Target image obtained is more truly reflected shooting main body, and shooting effect is good, user experience is good.
In order to realize above-described embodiment, the invention also provides a kind of image processing method, the executing subject of this method is
Image processing system.Image processing system includes: structured light projector 10, the first camera 20, second camera 30, image letter
Number processor 1130；Structured light projector 10 images cross central line altogether with first and is arranged successively, and total with second camera 30
Longitudinal centre line is arranged successively.Fig. 3 is the flow chart of the image processing method of one embodiment of the invention, as shown in figure 3, this method
Include:
Step 101, structured light projector 10 project default projection pattern structure light to shooting main body.
In one possible implementation, structured light projector 10 includes light source controller, array light source；
Light source controller shines for the luminance according to each light source for presetting projection pattern control array light source
State includes: illuminating state and OFF state；
Array light source presets projection pattern structure for issuing default projection pattern project structured light to shooting main body
Light is the structure light that the light that the light source of whole illuminating states issues is formed.
Preferably, image-signal processing system further includes light sensor；Light sensor, for obtaining external environmental light
Light intensity；Correspondingly, light source controller is used to control the luminous intensity of the light source of illuminating state according to light intensity.Citing
For, before step 101, by light sensor obtain external environmental light light intensity, then light source controller according to
Light intensity controls the luminous intensity of the light source of illuminating state, and then the luminous intensity of control structure light projector 10, guarantees good
Good shooting effect.
Step 102, the first camera 20 shoot first structure of the default projection pattern structure light by shooting main body modulation
Light image.
Step 103, second camera 30 shoot second structure of the default projection pattern structure light by shooting main body modulation
Light image.
Step 104, image-signal processor 1130 demodulate first structure light image to extract before depth value is greater than
Scape threshold value and the first depth of view information for being less than background threshold value demodulate the second structure light image to extract before depth value is greater than
Scape threshold value and the second depth of view information for being less than background threshold value, and image is carried out according to the first depth of view information and the second depth of view information
Fusion is to generate target image.
Specifically, image co-registration is carried out to generate mesh according to the first depth of view information and the second depth of view information in step 104
The possible implementation of logo image are as follows: image overlay region information is determined according to the first depth of view information and the first depth of view information；Root
Image segmentation is carried out to first structure light image according to image overlay region information, obtains the first overlapping image and the first non-overlap figure
Picture；Image segmentation is carried out to the second structure light image according to image overlay region information, obtains the second overlapping image and second non-heavy
Folded image；It chooses the first overlapping image and the second overlapping image carries out image co-registration to generate target image.
Method in this present embodiment is closed, wherein each step executes the concrete mode of operation in the related system
It is described in detail in embodiment, no detailed explanation will be given here.
Image processing method provided in an embodiment of the present invention, firstly, structured light projector 10 is default to shooting main body projection
Projection pattern structure light；Then, the first camera 20 shoots first of default projection pattern structure light by shooting main body modulation
Structure light image, second camera 30 shoot second structure light figure of the default projection pattern structure light by shooting main body modulation
Picture；Image-signal processor 1130 demodulates first structure light image to extract depth value greater than prospect threshold value and after being less than
First depth of view information of scape threshold value demodulates the second structure light image to extract depth value greater than prospect threshold value and after being less than
Second depth of view information of scape threshold value, and image co-registration is carried out to generate target according to the first depth of view information and the second depth of view information
Image.Target image obtained is more truly reflected shooting main body, and shooting effect is good, user experience is good.
It include image in above-mentioned terminal device the invention also provides a kind of terminal device to realize above-described embodiment
Processing circuit, image processing circuit can use hardware and or software component realization, it may include define ISP (Image Signal
Processing, image signal process) pipeline various processing units.Fig. 4 is terminal device according to an embodiment of the invention
In image processing circuit structural schematic diagram.As shown in figure 4, for purposes of illustration only, only showing relevant to the embodiment of the present invention
The various aspects of image processing techniques.
As indicated at 4, image processing circuit 110 includes imaging device 1110, ISP processor 1130 and control logic device
1140.Imaging device 1110 may include camera and structure light with one or more lens 1112, imaging sensor 1114
The projector 1116.Structured light projector 1116 is by structured light projection to measured object.Wherein, which can be laser strip
Line, Gray code, sine streak or, the speckle pattern etc. of random alignment.Imaging sensor 1114 captures projection to measured object shape
At structure light image, and structure light image is sent to ISP processor 1130, by ISP processor 1130 to structure light image
Carry out the depth information that demodulation obtains measured object.Meanwhile imaging sensor 1114 also can capture the color information of measured object.When
So, the structure light image and color information of measured object can also be captured respectively by two imaging sensors 1114.
Wherein, by taking pattern light as an example, ISP processor 1130 demodulates structure light image, specifically includes, from this
The speckle image that measured object is acquired in structure light image, by the speckle image of measured object and with reference to speckle image according to pre-defined algorithm
Image data calculating is carried out, each speckle point for obtaining speckle image on measured object is dissipated relative to the reference in reference speckle image
The moving distance of spot.The depth value of each speckle point of speckle image is calculated using trigonometry conversion, and according to the depth
Angle value obtains the depth information of measured object.
It is, of course, also possible to obtain the depth image by the method for binocular vision or based on the method for jet lag TOF
Information etc., it is not limited here, as long as the method that can obtain or be obtained by calculation the depth information of measured object belongs to this
The range that embodiment includes.
It, can quilt after the color information that ISP processor 1130 receives the measured object that imaging sensor 1114 captures
The corresponding image data of color information for surveying object is handled.ISP processor 1130 analyzes image data can with acquisition
For determining the image statistics of one or more control parameters of imaging device 1110.Imaging sensor 1114 may include color
Color filter array (such as Bayer filter), imaging sensor 1114 can be obtained is caught with each imaging pixel of imaging sensor 1114
The luminous intensity and wavelength information caught, and the one group of raw image data that can be handled by ISP processor 1130 is provided.
ISP processor 1130 handles raw image data pixel by pixel in various formats.For example, each image pixel can
Bit depth with 8,10,12 or 14 bits, ISP processor 1130 can carry out raw image data at one or more images
Reason operation, image statistics of the collection about image data.Wherein, image processing operations can be by identical or different bit depth
Precision carries out.
ISP processor 1130 can also receive pixel data from video memory 1120.Video memory 1120 can be storage
Independent private memory in a part of device device, storage equipment or electronic equipment, and may include DMA (Direct
Memory Access, direct memory access (DMA)) feature.
When receiving raw image data, ISP processor 1130 can carry out one or more image processing operations.
After ISP processor 1130 gets color information and the depth information of measured object, it can be merged, be obtained
3-D image.Wherein, it can be extracted by least one of appearance profile extracting method or contour feature extracting method corresponding
The feature of measured object.Such as pass through active shape model method ASM, active appearance models method AAM, Principal Component Analysis PCA, discrete
The methods of cosine transform method DCT, extracts the feature of measured object, it is not limited here.It will be extracted from depth information respectively again
The feature of measured object and the feature that measured object is extracted from color information carry out registration and Fusion Features processing.It herein refers to
Fusion treatment, which can be, directly combines the feature extracted in depth information and color information, is also possible to different images
In identical feature carry out weight setting after combine, can also have other amalgamation modes, finally according to fused feature, generate
3-D image.
The image data of 3-D image can be transmitted to video memory 1120, to carry out other place before shown
Reason.ISP processor 1130 from video memory 1120 receive processing data, and to the processing data progress original domain in and
Image real time transfer in RGB and YCbCr color space.The image data of 3-D image may be output to display 1160, for
User watches and/or is further processed by graphics engine or GPU (Graphics Processing Unit, graphics processor).
In addition, the output of ISP processor 1130 also can be transmitted to video memory 1120, and display 1160 can be from video memory
1120 read image data.In one embodiment, video memory 1120 can be configured to realize one or more frame bufferings
Device.In addition, the output of ISP processor 1130 can be transmitted to encoder/decoder 1150, so as to encoding/decoding image data.It compiles
The image data of code can be saved, and decompress before being shown in 1160 equipment of display.Encoder/decoder 1150 can
It is realized by CPU or GPU or coprocessor.
The image statistics that ISP processor 1130 determines, which can be transmitted, gives control logic device Unit 1140.Control logic device
1140 may include the processor and/or microcontroller for executing one or more routines (such as firmware), and one or more routines can root
According to received image statistics, the control parameter of imaging device 1110 is determined.
The following are realize image processing method with image processing techniques in Fig. 4:
Step 101, structured light projector projects default projection pattern structure light to shooting main body.
Step 102, the first camera shoots first structure light of the default projection pattern structure light by shooting main body modulation
Image.
Step 103, second camera shoots second structure light of the default projection pattern structure light by shooting main body modulation
Image.
Step 104, image-signal processor demodulates first structure light image to extract depth value greater than prospect threshold
Value and the first depth of view information for being less than background threshold value demodulate the second structure light image to extract depth value greater than prospect threshold
Value and the second depth of view information for being less than background threshold value, and image co-registration is carried out according to the first depth of view information and the second depth of view information
To generate target image.
In order to realize above-described embodiment, the present invention also proposes a kind of non-transitorycomputer readable storage medium, deposits thereon
Computer program is contained, can be realized image procossing as in the foregoing embodiment when the computer program is executed by processor
Method.
In the description of the present invention, it is to be understood that, term " first ", " second " are used for description purposes only, and cannot
It is interpreted as indication or suggestion relative importance or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " the
One ", the feature of " second " can explicitly or implicitly include one or more of the features.In the description of the present invention,
The meaning of " plurality " is two or more, unless otherwise specifically defined.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means that specific features described in conjunction with this embodiment or example or feature are contained in this
In at least one embodiment or example of invention.In the present specification, schematic expression of the above terms are necessarily directed to
It is identical embodiment or example.Moreover, the specific features or feature of description in any one or more embodiments or can show
It can be combined in any suitable manner in example.In addition, without conflicting with each other, those skilled in the art can be by this specification
Described in the features of different embodiments or examples and different embodiments or examples be combined.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.The more specific example (non-exhaustive list) of computer-readable medium include the following: there is the electricity of one or more wirings
Interconnecting piece (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable
Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.It, and in another embodiment, can be under well known in the art for example, if realized with hardware
Any one of column technology or their combination are realized: having a logic gates for realizing logic function to data-signal
Discrete logic, with suitable combinational logic gate circuit specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium
In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in a processing module
It is that each unit physically exists alone, can also be integrated in two or more units in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized and when sold or used as an independent product in the form of software function module, also can store in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only memory, disk or CD etc..Although having been shown and retouching above
The embodiment of the present invention is stated, it is to be understood that above-described embodiment is exemplary, and should not be understood as to limit of the invention
System, those skilled in the art can be changed above-described embodiment, modify, replace and become within the scope of the invention
Type.
Claims (8)
1. a kind of image processing system characterized by comprising structured light projector, the first camera, second camera, figure
As signal processor；
The structured light projector images cross central line altogether with described first and is arranged successively, and indulges altogether with the second camera
It is arranged successively to center line；
The structured light projector, for projecting default projection pattern structure light to shooting main body；
First camera, for shooting first knot of the default projection pattern structure light by the shooting main body modulation
Structure light image,
The second camera, for shooting second knot of the default projection pattern structure light by the shooting main body modulation
Structure light image；
Described image signal processor, for being demodulated to the first structure light image to extract depth value greater than prospect threshold
Value and the first depth of view information for being less than background threshold value demodulate second structure light image to extract before depth value is greater than
Scape threshold value and the second depth of view information for being less than background threshold value, and according to first depth of view information and second depth of view information
Image co-registration is carried out to generate target image；
Wherein, described image signal processor is specifically used for:
Image overlay region information is determined according to first depth of view information and second depth of view information；
According to described image overlay region information to the first structure light image carry out image segmentation, obtain the first overlapping image and
First non-overlapping images；
According to described image overlay region information to second structure light image carry out image segmentation, obtain the second overlapping image and
Second non-overlapping images；
It chooses the first overlapping image and the second overlapping image carries out image co-registration to generate target image.
2. the system as claimed in claim 1, which is characterized in that the structured light projector includes light source controller, array light
Source；
The light source controller, the luminous shape of each light source for controlling the array light source according to the default projection pattern
State, the luminance include: illuminating state and OFF state；
The array light source, for issuing default projection pattern project structured light to shooting main body, the default projection pattern
Structure light is the structure light that the light that the light source of whole illuminating states issues is formed.
3. system as claimed in claim 2, which is characterized in that further include: light sensor；
The light sensor, for obtaining the light intensity of external environmental light；
Correspondingly, the light source controller is used to control the luminous intensity of the light source of illuminating state according to the light intensity.
4. a kind of image processing method, which is characterized in that be applied to image processing system, described image processing system includes: knot
Structure light projector, the first camera, second camera, image-signal processor；
The structured light projector images cross central line altogether with described first and is arranged successively, and indulges altogether with the second camera
It is arranged successively to center line；
The structured light projector projects default projection pattern structure light to shooting main body；
First camera shoots first structure light of the default projection pattern structure light by the shooting main body modulation
Image,
The second camera shoots second structure light of the default projection pattern structure light by the shooting main body modulation
Image；
Described image signal processor the first structure light image is demodulated to extract depth value greater than prospect threshold value and
Less than the first depth of view information of background threshold value, second structure light image is demodulated to extract depth value greater than prospect threshold
Value and the second depth of view information for being less than background threshold value, and carried out according to first depth of view information and second depth of view information
Image co-registration is to generate target image；
Wherein, described that image co-registration is carried out to generate target image according to first depth of view information and second depth of view information
Include:
Image overlay region information is determined according to first depth of view information and second depth of view information；
According to described image overlay region information to the first structure light image carry out image segmentation, obtain the first overlapping image and
First non-overlapping images；
According to described image overlay region information to second structure light image carry out image segmentation, obtain the second overlapping image and
Second non-overlapping images；
It chooses the first overlapping image and the second overlapping image carries out image co-registration to generate target image.
5. method as claimed in claim 4, which is characterized in that the structured light projector includes light source controller, array light
Source；
The light source controller controls the luminance of each light source of the array light source, institute according to the default projection pattern
Stating luminance includes: illuminating state and OFF state；
The array light source issues on the default projection pattern project structured light to shooting main body, the default projection pattern knot
Structure light is the structure light that the light that the light source of whole illuminating states issues is formed.
6. method as claimed in claim 5, which is characterized in that described image processing system further include: light sensor；
The light sensor obtains the light intensity of external environmental light；
Correspondingly, the light source controller controls the luminous intensity of the light source of illuminating state according to the light intensity.
7. a kind of terminal device, which is characterized in that including memory and processor, stored in the memory computer-readable
Instruction, when described instruction is executed by the processor, so that the processor executes such as the described in any item figures of claim 4-6
As processing method.
8. a kind of non-transitorycomputer readable storage medium, is stored thereon with computer program, which is characterized in that the computer
Such as claim 4-6 described in any item image processing methods are realized when program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710676485.XA CN107493411B (en) | 2017-08-09 | 2017-08-09 | Image processing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710676485.XA CN107493411B (en) | 2017-08-09 | 2017-08-09 | Image processing system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107493411A CN107493411A (en) | 2017-12-19 |
| CN107493411B true CN107493411B (en) | 2019-09-13 |
Family
ID=60644075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710676485.XA Active CN107493411B (en) | 2017-08-09 | 2017-08-09 | Image processing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107493411B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108259722A (en) * | 2018-02-27 | 2018-07-06 | 厦门美图移动科技有限公司 | Imaging method, device and electronic equipment |
| CN108564540B (en) * | 2018-03-05 | 2020-07-17 | Oppo广东移动通信有限公司 | Image processing method and device for removing lens reflection in image and terminal equipment |
| CN109040556A (en) * | 2018-08-22 | 2018-12-18 | Oppo广东移动通信有限公司 | Imaging device and electronic equipment |
| CN109302569A (en) * | 2018-09-27 | 2019-02-01 | 维沃移动通信有限公司 | A kind of image imaging method and device of mobile terminal |
| CN113703248B (en) * | 2021-08-11 | 2022-09-09 | 深圳市睿识科技有限公司 | 3D structured light module and depth map point cloud image acquisition method based on same |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102760234A (en) * | 2011-04-14 | 2012-10-31 | 财团法人工业技术研究院 | Depth image acquisition device, system and method |
| CN104166972A (en) * | 2013-05-17 | 2014-11-26 | 中兴通讯股份有限公司 | Terminal and method for realizing image processing |
| CN104634276A (en) * | 2015-02-12 | 2015-05-20 | 北京唯创视界科技有限公司 | Three-dimensional measuring system, photographing device, photographing method, depth calculation method and depth calculation device |
| CN105096283A (en) * | 2014-04-29 | 2015-11-25 | 华为技术有限公司 | Panoramic image acquisition method and device |
| CN105407280A (en) * | 2015-11-11 | 2016-03-16 | 广东欧珀移动通信有限公司 | Panoramic image synthesizing method and system |
| CN106572340A (en) * | 2016-10-27 | 2017-04-19 | 深圳奥比中光科技有限公司 | Camera shooting system, mobile terminal and image processing method |
| CN106954058A (en) * | 2017-03-09 | 2017-07-14 | 深圳奥比中光科技有限公司 | Depth image obtains system and method |
-
2017
- 2017-08-09 CN CN201710676485.XA patent/CN107493411B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102760234A (en) * | 2011-04-14 | 2012-10-31 | 财团法人工业技术研究院 | Depth image acquisition device, system and method |
| CN104166972A (en) * | 2013-05-17 | 2014-11-26 | 中兴通讯股份有限公司 | Terminal and method for realizing image processing |
| CN105096283A (en) * | 2014-04-29 | 2015-11-25 | 华为技术有限公司 | Panoramic image acquisition method and device |
| CN104634276A (en) * | 2015-02-12 | 2015-05-20 | 北京唯创视界科技有限公司 | Three-dimensional measuring system, photographing device, photographing method, depth calculation method and depth calculation device |
| CN105407280A (en) * | 2015-11-11 | 2016-03-16 | 广东欧珀移动通信有限公司 | Panoramic image synthesizing method and system |
| CN106572340A (en) * | 2016-10-27 | 2017-04-19 | 深圳奥比中光科技有限公司 | Camera shooting system, mobile terminal and image processing method |
| CN106954058A (en) * | 2017-03-09 | 2017-07-14 | 深圳奥比中光科技有限公司 | Depth image obtains system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107493411A (en) | 2017-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107395974B (en) | Image processing system and method | |
| CN107493411B (en) | Image processing system and method | |
| CN106504284B (en) | A kind of depth picture capturing method combined based on Stereo matching with structure light | |
| CN107483845B (en) | Photographic method and its device | |
| CN107370951A (en) | Image processing system and method | |
| US9392262B2 (en) | System and method for 3D reconstruction using multiple multi-channel cameras | |
| CN107734267B (en) | Image processing method and device | |
| TWI584634B (en) | Electronic apparatus and method of generating depth map | |
| WO2019047985A1 (en) | Image processing method and device, electronic device, and computer-readable storage medium | |
| US20090324017A1 (en) | Capturing and processing facial motion data | |
| CN108765542A (en) | Image rendering method, electronic equipment and computer readable storage medium | |
| CN107493428A (en) | Filming control method and device | |
| CN107623817B (en) | Video background processing method, device and mobile terminal | |
| CN107529096A (en) | Image processing method and device | |
| CN107610080B (en) | Image processing method and apparatus, electronic apparatus, and computer-readable storage medium | |
| CN107451561A (en) | Iris recognition light compensation method and device | |
| CN107707839A (en) | Image processing method and device | |
| CN107493412B (en) | Image processing system and method | |
| CN107493427A (en) | Focusing method, device and the mobile terminal of mobile terminal | |
| Starck et al. | The multiple-camera 3-d production studio | |
| CN107481317A (en) | The facial method of adjustment and its device of face 3D models | |
| CN107734264B (en) | Image processing method and device | |
| CN107705278B (en) | Dynamic effect adding method and terminal equipment | |
| CN107517346A (en) | Photographic method, device and mobile device based on structure light | |
| CN110378944A (en) | Depth map processing method, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant after: OPPO Guangdong Mobile Communications Co., Ltd. Address before: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant before: Guangdong OPPO Mobile Communications Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |