CN110276831A - Constructing method and device, equipment, the computer readable storage medium of threedimensional model - Google Patents

Abstract

This application involves a kind of constructing method of threedimensional model, device, terminal device and computer readable storage mediums.Method includes obtaining visible light figure, generates center weight figure corresponding with visible light figure, wherein weighted value represented by center weight figure is gradually reduced from center to edge；It will be seen that light figure and center weight figure are input in subject detection model, obtain body region confidence level figure, wherein, subject detection model is the model that the visible light figure previously according to Same Scene, center weight figure and the corresponding main body exposure mask figure marked are trained；The target subject in the visible light figure is determined according to body region confidence level figure；Obtain the corresponding depth information of target subject；According to target subject and the corresponding depth information of target subject, three-dimensionalreconstruction is carried out to target subject, the step of returning to acquisition visible light figure improves the accuracy rate of threedimensional model construction until obtaining the corresponding threedimensional model of target subject with the visible light figure for obtaining different acquisition angle.

Description

Constructing method and device, equipment, the computer readable storage medium of threedimensional model

Technical field

This application involves field of computer technology, constructing methods and device more particularly to a kind of threedimensional model, terminal Equipment, computer readable storage medium.

Background technique

With the development of image technology, people are increasingly accustomed to through image capture devices such as cameras on electronic equipment Image or video are shot, various information are recorded, is received extensive attention since the sense of reality of three-dimensional image procossing is stronger.

Traditional threedimensional model is often subject to the influence of the people or object of surrounding, leads to three-dimensional when carrying out three-dimensional construction The construction accuracy rate of model is low.

Summary of the invention

The embodiment of the present application provides constructing method, device, terminal device, the computer-readable storage medium of a kind of threedimensional model Matter obtains accurately body region confidence level figure according to center weight figure and subject detection model, to accurately identify image In target subject by the depth information of target subject, realize the corresponding three-dimensional mould of target subject in threedimensional model construction The accurate building of type, improves the accuracy rate of threedimensional model construction.

A kind of constructing method of threedimensional model, which comprises

Visible light figure is obtained, generates center weight figure corresponding with the visible light figure, wherein the center weight figure institute The weighted value of expression is gradually reduced from center to edge；

The visible light figure and the center weight figure are input in subject detection model, body region confidence level is obtained Figure, wherein the subject detection model is the visible light figure previously according to Same Scene, center weight figure and corresponding has marked The model that is trained of main body exposure mask figure；

The target subject in the visible light figure is determined according to the body region confidence level figure；

Obtain the corresponding depth information of the target subject；

According to the target subject and the corresponding depth information of target subject, three-dimensionalreconstruction is carried out to the target subject, Described the step of obtaining visible light figure is returned to obtain the visible light figure of different acquisition angle, until obtaining the target subject pair The threedimensional model answered.

A kind of constructing devices of threedimensional model, described device include:

Processing module generates center weight figure corresponding with the visible light figure, wherein institute for obtaining visible light figure Weighted value represented by center weight figure is stated to be gradually reduced from center to edge；

Detection module is obtained for the visible light figure and the center weight figure to be input in subject detection model Body region confidence level figure, wherein the subject detection model is visible light figure, the center weight figure previously according to Same Scene And the model that the corresponding main body exposure mask figure marked is trained；

Target subject determining module, for determining the target in the visible light figure according to the body region confidence level figure Main body；

Threedimensional model building block, for obtaining the corresponding depth information of the target subject, according to the target subject Depth information corresponding with target subject carries out three-dimensionalreconstruction to the target subject, returns to the step for obtaining visible light figure Suddenly to obtain the visible light figure of different acquisition angle, until obtaining the corresponding threedimensional model of the target subject.

A kind of terminal device, including memory and processor store computer program, the calculating in the memory When machine program is executed by the processor, so that the processor executes following steps:

Visible light figure is obtained, generates center weight figure corresponding with the visible light figure, wherein the center weight figure institute The weighted value of expression is gradually reduced from center to edge；

The visible light figure and the center weight figure are input in subject detection model, body region confidence level is obtained Figure, wherein the subject detection model is the visible light figure previously according to Same Scene, center weight figure and corresponding has marked The model that is trained of main body exposure mask figure；

The target subject in the visible light figure is determined according to the body region confidence level figure；

Obtain the corresponding depth information of the target subject；

According to the target subject and the corresponding depth information of target subject, three-dimensionalreconstruction is carried out to the target subject, Described the step of obtaining visible light figure is returned to obtain the visible light figure of different acquisition angle, until obtaining the target subject pair The threedimensional model answered.

A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor Following steps are realized when row:

Visible light figure is obtained, generates center weight figure corresponding with the visible light figure, wherein the center weight figure institute The weighted value of expression is gradually reduced from center to edge；

The visible light figure and the center weight figure are input in subject detection model, body region confidence level is obtained Figure, wherein the subject detection model is the visible light figure previously according to Same Scene, center weight figure and corresponding has marked The model that is trained of main body exposure mask figure；

The target subject in the visible light figure is determined according to the body region confidence level figure；

Obtain the corresponding depth information of the target subject；

According to the target subject and the corresponding depth information of target subject, three-dimensionalreconstruction is carried out to the target subject, Described the step of obtaining visible light figure is returned to obtain the visible light figure of different acquisition angle, until obtaining the target subject pair The threedimensional model answered.

Constructing method, device, terminal device, the computer readable storage medium of above-mentioned threedimensional model, it is visible by obtaining Light figure generates center weight figure corresponding with visible light figure, wherein weighted value represented by center weight figure is from center to edge It is gradually reduced；It will be seen that light figure and center weight figure are input in subject detection model, body region confidence level figure is obtained, In, subject detection model is that the visible light figure previously according to Same Scene, center weight figure and the corresponding main body marked are covered The model that film figure is trained；The target subject in visible light figure is determined according to body region confidence level figure；Obtain target The corresponding depth information of main body；According to target subject and the corresponding depth information of target subject, Three-dimensional Gravity is carried out to target subject Structure, returns to the step of obtaining visible light figure to obtain the visible light figure of different acquisition angle, corresponding until obtaining target subject Threedimensional model can allow the object of picture centre to be easier to be detected, using trained using visible using center weight figure The subject detection model that the training such as light figure, center weight figure and main body exposure mask figure obtains, can more accurately identify visible Target subject in light figure, by the depth information of target subject, realizes target subject corresponding three in threedimensional model construction The accurate building of dimension module, also can precisely identify target subject there are interfering object, to improve target subject The accuracy rate of corresponding threedimensional model construction.

Detailed description of the invention

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 is the internal structure block diagram of terminal device in one embodiment；

Fig. 2 is the flow chart of the constructing method of threedimensional model in one embodiment；

Fig. 3 is the schematic diagram of the corresponding threedimensional model of target subject in one embodiment；

Fig. 4 is the stream for determining the target subject in the visible light figure in one embodiment according to the body region confidence level figure Cheng Tu；

Fig. 5 is the schematic network structure of subject detection model in one embodiment；

Fig. 6 is subject detection effect diagram in one embodiment；

Fig. 7 is the structural block diagram of the constructing devices of threedimensional model in one embodiment；

Fig. 8 is the internal structure chart of terminal device in one embodiment.

Specific embodiment

It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and It is not used in restriction the application.

The constructing method of threedimensional model in the embodiment of the present application can be applied to terminal device.The terminal device can for Computer equipment, personal digital assistant, tablet computer, smart phone, wearable device of camera etc..Taking the photograph in terminal device Picture head will do it auto-focusing when shooting image, to guarantee the image clearly of shooting.

It in one embodiment, may include image processing circuit in above-mentioned terminal device, image processing circuit can use Hardware and or software component is realized, it may include defines ISP (Image Signal Processing, image signal process) pipeline Various processing units.Fig. 1 is the schematic diagram of image processing circuit in one embodiment.As shown in Figure 1, for purposes of illustration only, only The various aspects of image processing techniques relevant to the embodiment of the present application are shown.

As shown in Figure 1, image processing circuit includes the first ISP processor 130, the 2nd ISP processor 140 and control logic Device 150.First camera 110 includes one or more first lens 112 and the first imaging sensor 114.First image sensing Device 114 may include colour filter array (such as Bayer filter), and the first imaging sensor 114 can be obtained with the first imaging sensor The luminous intensity and wavelength information that 114 each imaging pixel captures, and one group for being handled by the first ISP processor 130 is provided Image data.Second camera 120 includes one or more second lens 122 and the second imaging sensor 124.Second image passes Sensor 124 may include colour filter array (such as Bayer filter), and the second imaging sensor 124 can be obtained with the second image sensing The luminous intensity and wavelength information that each imaging pixel of device 124 captures, and can be handled by the 2nd ISP processor 140 one is provided Group image data.

First image transmitting of the first camera 110 acquisition is handled to the first ISP processor 130, the first ISP processing It, can be by statistical data (brightness of such as image, the contrast value of image, the face of image of the first image after device 130 handles the first image Color etc.) it is sent to control logic device 150, control logic device 150 can determine the control ginseng of the first camera 110 according to statistical data Number, so that the first camera 110 can carry out the operation such as auto-focusing, automatic exposure according to control parameter.First image is by the One ISP processor 130 can store after being handled into video memory 160, and the first ISP processor 130 can also read figure As the image that stores in memory 160 is with to handling.In addition, the first image can after ISP processor 130 is handled It is sent directly to display 170 to be shown, display 170 can also read the image in video memory 160 to be shown Show.

Wherein, the first ISP processor 130 handles image data pixel by pixel in various formats.For example, each image slices Element can have the bit depth of 8,10,12 or 14 bits, and the first ISP processor 130 can carry out one or more figures to image data Statistical information as processing operation, collection about image data.Wherein, image processing operations can be by identical or different bit depth Precision carries out.

Video memory 160 can be independent dedicated in a part, storage equipment or terminal device of memory device Memory, and may include DMA (Direct Memory Access, direct direct memory access (DMA)) feature.

When receiving from the first 114 interface of imaging sensor, the first ISP processor 130 can carry out one or more Image processing operations, such as time-domain filtering.Image data that treated can be transmitted to video memory 160, to be shown it It is preceding to carry out other processing.First ISP processor 130 receives processing data from video memory 160, and to the processing data Carry out the image real time transfer in RGB and YCbCr color space.Treated that image data is exportable for first ISP processor 130 To display 170, so that user watches and/or by graphics engine or GPU (Graphics Processing Unit, at figure Reason device) it is further processed.In addition, the output of the first ISP processor 130 also can be transmitted to video memory 160, and display 170 can read image data from video memory 160.In one embodiment, video memory 160 can be configured to realization one A or multiple frame buffers.

The statistical data that first ISP processor 130 determines can be transmitted to control logic device 150.For example, statistical data can wrap Include automatic exposure, automatic white balance, automatic focusing, flicker detection, black level compensation, 112 shadow correction of the first lens etc. first 114 statistical information of imaging sensor.Control logic device 150 may include the processor for executing one or more routines (such as firmware) And/or microcontroller, one or more routines can statistical data based on the received, determine the control parameter of the first camera 110 And the first ISP processor 130 control parameter.For example, the control parameter of the first camera 110 may include gain, spectrum assignment The time of integration, stabilization parameter, flash of light control parameter, 112 control parameter of the first lens (such as focus or zoom focal length) or The combination etc. of these parameters.ISP control parameter may include for automatic white balance and color adjustment (for example, in RGB process phase Between) 112 shadow correction parameter of gain level and color correction matrix and the first lens.

Similarly, the second image transmitting that second camera 120 acquires is handled to the 2nd ISP processor 140, and second After ISP processor 140 handles the first image, can by the statistical data of the second image (brightness of such as image, image contrast value, The color etc. of image) it is sent to control logic device 150, control logic device 150 can determine second camera 120 according to statistical data Control parameter, so that second camera 120 can carry out auto-focusing, the operation such as automatic exposure according to control parameter.Second figure As that can store after the 2nd ISP processor 140 is handled into video memory 160, the 2nd ISP processor 140 can also To read the image stored in video memory 160 with to handling.In addition, the second image is carried out by ISP processor 140 It can be sent directly to display 170 after processing and shown that display 170 can also read the image in video memory 160 To be shown.Second camera 120 and the 2nd ISP processor 140 also may be implemented such as the first camera 110 and the first ISP Treatment process described in processor 130.

In one embodiment, the first camera 110 can be colour imagery shot, and second camera 120 can be TOF (Time Of Flight, flight time) camera or structure light video camera head.TOF camera can obtain TOF depth map, structure light video camera head Structure light depth map can be obtained.First camera 110 and second camera 120 can be colour imagery shot.Pass through two colours Camera obtains binocular depth figure.First ISP processor 130 and the 2nd ISP processor 140 can be same ISP processor.

First camera 110 and second camera 120 shoot Same Scene and respectively obtain visible light figure and depth map, can Light-exposed figure and depth map are sent to ISP processor.ISP processor can according to camera calibration parameter to visible light figure and depth map into Row registration, keeps the visual field completely the same；Then center weight figure corresponding with visible light figure is regenerated, wherein the center weight The represented weighted value of figure is gradually reduced from center to edge；It will be seen that light figure and center weight figure are input to trained main body In detection model, body region confidence level figure is obtained, the target master in visible light figure is determined further according to body region confidence level figure Body；It can also will be seen that light figure, depth map and center weight figure are input in trained subject detection model, obtain body region Confidence level figure determines the target subject in visible light figure further according to body region confidence level figure.It can be allowed using center weight figure It is easier to be detected positioned at the object of picture centre, can be allowed using depth map and be easy to be detected apart from the closer object of camera It surveys, improves the accuracy of subject detection.In threedimensional model construction, by the depth information of target subject, target master is realized The accurate building of the corresponding threedimensional model of body, also can precisely identify target subject there are interfering object, to mention The accuracy rate of the corresponding threedimensional model construction of high target subject.

Fig. 2 is the flow chart of the constructing method of threedimensional model in one embodiment.As shown in Fig. 2, a kind of threedimensional model Constructing method can be applied in the terminal device in Fig. 1, comprising:

Step 202, visible light figure is obtained.

Wherein, when subject detection (salient object detection) is referred in face of a scene, automatically to sense Interest region handled and selectivity ignore region of loseing interest in.Area-of-interest is known as body region.Visible light figure is Refer to RGB (Red, Green, Blue) image.Any scene can be shot by colour imagery shot obtains color image, i.e. RGB image. What the visible light figure can be locally stored for terminal device, can also be other equipment storage, or it is stored from network, It can be also terminal device captured in real-time, it is without being limited thereto.

Specifically, the ISP processor of terminal device or central processing unit can be obtained from local or other equipment or network Visible light figure, or a scene is shot by camera and obtains visible light figure.

Step 204, center weight figure corresponding with the visible light figure is generated, wherein power represented by the center weight figure Weight values are gradually reduced from center to edge.

Wherein, center weight figure refers to the figure for recording the weighted value of each pixel in visible light figure.Center weight The weighted value recorded in figure is gradually reduced from center to four sides, i.e., center weight is maximum, is gradually reduced again to four side rights.In The weighted value of picture centre pixel to the image edge pixels point of heart weight map characterization visible light figure is gradually reduced.

ISP processor or central processing unit can generate corresponding center weight figure according to the size of visible light figure.In this Weighted value represented by heart weight map is gradually reduced from center to four sides.Center weight figure can be used Gaussian function or using one Rank equation or second-order equation generate.The Gaussian function can be two-dimensional Gaussian function.

Step 206, the visible light figure and center weight figure are input in subject detection model, obtain body region confidence Degree figure, wherein subject detection model is the visible light figure previously according to Same Scene, center weight figure and corresponding has marked The model that main body exposure mask figure is trained.

Wherein, subject detection model is to acquire a large amount of training data in advance, and it includes initial that training data, which is input to, What the subject detection model of network weight was trained.Every group of training data include the corresponding visible light figure of Same Scene, Center weight figure and the main body exposure mask figure marked.Wherein, it is seen that the subject detection mould of light figure and center weight figure as training The input of type, the true value that main body exposure mask (mask) figure marked is obtained as the subject detection model desired output of training (ground truth).Main body exposure mask figure is the image filters template of main body in image for identification, can with shielded image its His part, filters out the main body in image.Subject detection model can training can the various main bodys of recognition detection, as people, flower, cat, Dog, background etc..

Specifically, the visible light figure and center weight figure can be input to subject detection by ISP processor or central processing unit In model, carry out detecting available body region confidence level figure.Body region confidence level figure is for recording which main body belongs to The probability for the main body that kind can identify, such as it is 0.8 that some pixel, which belongs to the probability of people, colored probability is 0.1, the probability of background It is 0.1.

Step 208, the target subject in the visible light figure is determined according to the body region confidence level figure.

Wherein, main body refers to various objects, such as people, flower, cat, dog, ox, blue sky, white clouds, background.Target subject refers to The main body needed can select as needed.

Specifically, ISP processor or central processing unit can choose confidence level highest or secondary according to body region confidence level figure The high main body as in visible light figure, a main body if it exists, then using the main body as target subject；Multiple masters if it exists Body can select wherein one or more main bodys as target subject according to configuration information or depth information.In one embodiment, Distance of each main body apart from camera terminal is determined according to the corresponding depth information of each main body, it will be apart from the smallest main body conduct Target subject.

Step 210, the corresponding depth information of target subject is obtained, is believed according to target subject and the corresponding depth of target subject Breath carries out three-dimensionalreconstruction to target subject, and return step 202 is to obtain the visible light figure of different acquisition angle, until obtaining mesh Mark the corresponding threedimensional model of main body.

Wherein, for target subject there are accurate profile, each pixel that need to only obtain in target subject profile is corresponding Depth information can obtain the corresponding depth information of target subject from the depth map that camera is shot, and wherein depth map obtains Mode is unlimited.

Specifically, three-dimensionalreconstruction is carried out to target subject according to target subject and the corresponding depth information of target subject, into The mode of row three-dimensionalreconstruction is not construed as limiting.Wherein depth information characterize each pixel in target subject away from capture apparatus away from From can determine the coordinate of the Z axis in three dimensions of each pixel in target subject according to depth information, thus to target subject Carry out three-dimensionalreconstruction.When carrying out three-dimensionalreconstruction, the pixel of same depth information can choose reference pixel in approximately the same plane Point, using the corresponding depth value of reference image vegetarian refreshments as reference depth value, the corresponding depth value of more other pixels and with reference to deep The size relation of angle value, so that it is determined that other pixels are relative to reference image vegetarian refreshments in the position of three-dimensional space.Such as target master The depth value of a certain pixel is 10 on body, the depth value of neighbor pixel be 12, then the point that this object is 10 with depth value As reference image vegetarian refreshments, then neighbor pixel can recessed 12-10=2 depth, target subject three-dimensionalreconstruction is exactly in target subject Its opposite concave-convex information is assigned in corresponding body surfaces.Since current visible light figure only has taken one of target subject Point, so need to obtain the visible light figure of other acquisition angles, if it is the visible light figure of captured in real-time, then convertible acquisition angle Degree, shoots the various pieces of target subject, and obtain the corresponding depth information of various pieces, thus to complete target Main body carries out three-dimensionalreconstruction.It, then can from directly acquire other shooting angle if it is the visible light figure of acquisition shot Light-exposed figure carries out three-dimensionalreconstruction to complete target subject, until obtaining the corresponding threedimensional model of target subject.As shown in figure 3, For the schematic diagram of the corresponding threedimensional model of target subject in one embodiment, built up.

The constructing method of threedimensional model in the present embodiment is generated corresponding with visible light figure by obtaining visible light figure Center weight figure, wherein weighted value represented by center weight figure is gradually reduced from center to edge；It will be seen that light figure and center Weight map is input in subject detection model, obtains body region confidence level figure, wherein subject detection model is previously according to same The model that visible light figure, center weight figure and the corresponding main body exposure mask figure marked of one scene are trained；According to Body region confidence level figure determines the target subject in visible light figure；Obtain the corresponding depth information of target subject；According to target Main body and the corresponding depth information of target subject, to target subject carry out three-dimensionalreconstruction, return obtain visible light figure the step of with The visible light figure for obtaining different acquisition angle can be with using center weight figure until obtaining the corresponding threedimensional model of target subject It allows the object of picture centre to be easier to be detected, utilizes visible light figure, center weight figure and main body exposure mask figure using trained Deng the subject detection model that training obtains, the target subject in visible light figure can be more accurately identified, in threedimensional model When construction, by the depth information of target subject, realizes the accurate building of the corresponding threedimensional model of target subject, interfered existing Also target subject can be precisely identified in the case where object, to improve the accuracy rate of the corresponding threedimensional model construction of target subject.

In one embodiment, as shown in figure 4, step 208, comprising:

Step 208A handles the body region confidence level figure, obtains main body exposure mask figure.

Specifically, there are some confidence levels in body region confidence level figure lower, scattered point, can pass through ISP processor Or central processing unit is filtered processing to body region confidence level figure, obtains main body exposure mask figure.The filtration treatment, which can be used, matches Confidence threshold value is set, the pixel by confidence value in body region confidence level figure lower than confidence threshold value filters.The confidence level Self-adapting confidence degree threshold value can be used in threshold value, can also use fixed threshold, can also use the corresponding threshold value of subregion configuration of territory.

Step 208B detects the visible light figure, determines the highlight area in the visible light figure.

Wherein, highlight area refers to that brightness value is greater than the region of luminance threshold.

Specifically, ISP processor or central processing unit carry out highlight detection to visible light figure, and screening obtains brightness value and is greater than The target pixel points of luminance threshold obtain highlight area using Connected area disposal$ to target pixel points.

Step 208C is determined and is disappeared in the visible light figure according to the highlight area and the main body exposure mask figure in the visible light figure Except the target subject of bloom.

Specifically, ISP processor or central processing unit can will be seen that highlight area and the main body exposure mask figure in light figure are done The target subject that bloom is eliminated in visible light figure is calculated in Difference Calculation or logical AND.

In the present embodiment, filtration treatment is done to body region confidence level figure and obtains main body exposure mask figure, improves body region The reliability of confidence level figure detects visible light figure to obtain highlight area, is then handled, can be obtained with main body exposure mask figure To the target subject for eliminating bloom, filter is individually used to carry out for bloom, the highlight regions for influencing main body accuracy of identification Processing improves the precision and accuracy of main body identification.

In one embodiment, step 208A, comprising: self-adapting confidence degree threshold value is carried out to the body region confidence level figure Filtration treatment obtains main body exposure mask figure.

Wherein, self-adapting confidence degree threshold value refers to confidence threshold value.Self-adapting confidence degree threshold value can set for local auto-adaptive Confidence threshold.The local auto-adaptive confidence threshold value is to determine the pixel according to the pixel Distribution value of the field block of pixel Binaryzation confidence threshold value on position.Higher, the brightness of the binaryzation confidence threshold value configuration of the higher image-region of brightness The binarization threshold confidence level of lower image-region configures lower.

In one embodiment, the configuration process of self-adapting confidence degree threshold value includes: when the brightness value of pixel is greater than the One brightness value then configures the first confidence threshold value, when the brightness value of pixel is less than the second brightness value, then configures the second confidence level Threshold value then configures third confidence threshold value when the brightness value of pixel is greater than the second brightness value and less than the first brightness value, In, the second brightness value is less than or equal to the first brightness value, and the second confidence threshold value is less than third confidence threshold value, third confidence level Threshold value is less than the first confidence threshold value.

In one embodiment, the configuration process of self-adapting confidence degree threshold value includes: when the brightness value of pixel is greater than the One brightness value then configures the first confidence threshold value, when pixel brightness value be less than or equal to the first brightness value, then configure second Confidence threshold value, wherein the second brightness value is less than or equal to the first brightness value, and the second confidence threshold value is less than the first confidence level threshold Value.

It, will be in body region confidence level figure when carrying out the processing of self-adapting confidence degree threshold filtering to body region confidence level figure The confidence value of each pixel then retains the pixel more than or equal to confidence threshold value compared with corresponding confidence threshold value, Then remove the pixel less than confidence threshold value.

In one embodiment, this carries out the processing of self-adapting confidence degree threshold filtering to the body region confidence level figure, obtains To main body exposure mask figure, comprising:

The processing of self-adapting confidence degree threshold filtering is carried out to the body region confidence level figure, obtains binaryzation exposure mask figure；It is right The binaryzation exposure mask figure carries out Morphological scale-space and guiding filtering processing, obtains main body exposure mask figure.

Specifically, ISP processor or central processing unit are by body region confidence level figure according to self-adapting confidence degree threshold value mistake After filter processing, the confidence value of the pixel of reservation is indicated using 1, the confidence value of the pixel removed is indicated using 0, is obtained To binaryzation exposure mask figure.

Morphological scale-space may include corrosion and expansion.Etching operation first can be carried out to binaryzation exposure mask figure, then be expanded Operation removes noise；Filtering processing is guided to the binaryzation exposure mask figure after Morphological scale-space again, realizes edge filter behaviour Make, obtains the main body exposure mask figure of edge extracting.

The noise for the main body exposure mask figure that can be guaranteed by Morphological scale-space and guiding filtering processing is few or does not make an uproar Point, edge are softer.

In one embodiment, this determines that this is visible according to highlight area and the main body exposure mask figure in the visible light figure The target subject of bloom is eliminated in light figure, comprising: do the highlight area in the visible light figure at difference with the main body exposure mask figure Reason, the target subject for the bloom that is eliminated.

Specifically, ISP processor or central processing unit do the highlight area in the visible light figure with the main body exposure mask figure Corresponding pixel value subtracts each other in difference processing, i.e. visible light figure and main body exposure mask figure, obtains the target subject in the visible light figure. The target subject of removal bloom is obtained by difference processing, calculation is simple.

In one embodiment, which includes the input layer being sequentially connected, middle layer and output layer.It should Visible light figure and the center weight figure are input in subject detection model, comprising: the visible light figure is acted on subject detection mould The input layer of type；The center weight figure is acted on to the output layer of the subject detection model.

Deep learning network model can be used in subject detection model.The deep learning network model may include being sequentially connected Input layer, middle layer and output layer.Middle layer can be one layer or at least two layers of network structure.Visible light figure is from subject detection mould The input layer of type inputs, that is, acts on the input layer of subject detection model.Output layer of the center weight figure in subject detection model Input, that is, act on the output layer of subject detection model.Center weight figure is acted on to the output layer of subject detection model, it can be with Influence of other layers to weight map for reducing subject detection model allows based on the object in picture center is more easier to be detected Body.

In one embodiment, it includes: acquisition and visible light figure that the corresponding depth information of target subject is obtained in step 210 Corresponding depth map；Depth map includes at least one of TOF depth map, binocular depth figure and structure light depth map, to visible light Figure and depth map carry out registration process, visible light figure and depth map after being registrated, according to target subject institute in visible light figure Region the corresponding depth information of target subject is determined from the depth map after registration.

Wherein, depth map refers to figure including depth information.Same field is shot by depth camera or binocular camera Scape obtains corresponding depth map.Depth camera can be structure light video camera head or TOF camera.Depth map can be structure optical depth At least one of figure, TOF depth map and binocular depth figure.

Specifically, ISP processor or central processing unit can be shot Same Scene by camera and obtain visible light figure and right Then the depth map answered is registrated visible light figure and depth map using camera calibration parameter, the visible light after being registrated Figure and depth map.In visible light figure and depth map after registration, a pixel in each visible light figure is in depth map There are matched pixels, thus the corresponding pixel in depth map of the region where target subject can be obtained according to matching relationship Point obtains the corresponding depth value of target subject according to the pixel value of pixel.

It in one embodiment, can simulations depth map when can not shoot to obtain depth map.It emulates in depth map The depth value of each pixel can be preset value.In addition, the depth value of each pixel in emulation depth map can correspond to not Same preset value.

In one embodiment, comprising: be input to visible light figure, the depth map and center weight figure after the registration In subject detection model, body region confidence level figure is obtained；Wherein, the subject detection model be previously according to Same Scene can The model that light-exposed figure, depth map, center weight figure and the corresponding main body exposure mask figure marked are trained.

Wherein, subject detection model is to acquire a large amount of training data in advance, and it includes initial that training data, which is input to, What the subject detection model of network weight was trained.Every group of training data include the corresponding visible light figure of Same Scene, Depth map, center weight figure and the main body exposure mask figure marked.Wherein, it is seen that the main body of light figure and center weight figure as training The input of detection model, the true value that the main body exposure mask figure marked is obtained as the subject detection model desired output of training. Main body exposure mask figure is the image filters template of main body in image for identification, can filter out figure with the other parts of shielded image Main body as in.Subject detection model can training can the various main bodys of recognition detection, such as people, flower, cat, dog, background.

In the present embodiment, using depth map and center weight figure as the input of subject detection model, depth map can use Depth information allow apart from the closer object of camera be easier be detected, four sides big using center weight in center weight figure The small center attention mechanism of weight allows the object of picture centre to be easier to be detected, and introduces depth map realization and does depth to main body Feature enhancing is spent, center weight figure is introduced and attention feature enhancing in center is done to main body, can not only accurately identify simple scenario Under target subject, more substantially increase the main body recognition accuracy under complex scene, introducing depth map can solve traditional mesh Mark the detection method problem poor to the ever-changing robustness of objective function of natural image.Simple scenario refers to that main body is single, background The not high scene of region contrast.

In one embodiment, the training method of the subject detection model, comprising: obtain Same Scene visible light figure, Depth map and the main body exposure mask figure marked；Generate center weight figure corresponding with the visible light figure, wherein the center weight figure Represented weighted value is gradually reduced from center to edge；The visible light figure is acted on into the inspection of the main body comprising initial network weight The depth map and the center weight figure are acted on the output layer of initial subject detection model, by this by the input layer for surveying model The true value that the main body exposure mask figure marked is exported as the subject detection model examines the main body that this includes initial network weight It surveys model to be trained, obtains the target network weight of the subject detection model.

Collect visible light figure, depth map and the corresponding main body exposure mask figure marked of a scene.To visible light figure The mark that semantic class is carried out with depth map marks the main body of the inside.A large amount of visible light figure is collected, COCO data are then based on The foreground target figure and simple Background of concentration are merged to obtain the image of a large amount of solid background or simple background, as Trained visible light figure.It include large number of foreground target in COCO data set.

The network structure of subject detection model uses the framework based on mobile-Unet, and increases layer in the part decoder Between bridge joint, make high-level semantics feature up-sampling when more fully transmit.Center weight figure acts on main body monitoring model Output layer, introduce center attention mechanism, allow the object in picture center to be easier to be detected as main body.

The network structure of subject detection model includes input layer, convolutional layer (conv), pond layer (pooling), bilinearity Interpolated layer (Bilinear Up sampling), convolution feature articulamentum (concat+conv), output layer etc..It is inserted in bilinearity It is worth between layer and convolution feature articulamentum and bridge joint is realized using deconvolution+add (superposition of deconvolution feature) operation, makes High-level semantics feature is obtained more fully to transmit in up-sampling.Convolutional layer, pond layer, bilinear interpolation layer, the connection of convolution feature Layer etc. can based on detection model middle layer.

Initial network weight refers to each layer of initial weight of the deep learning network model of initialization.Target network power Refer to each layer of weight of the deep learning network model for being able to detect image subject that training obtains again.Default instruction can be passed through Practice number and obtain target network weight, the loss function of deep learning network model also can be set.When training obtains loss letter When numerical value is less than loss threshold value, using the current network weight of subject detection model as target network weight.

Fig. 5 is the schematic network structure of subject detection model in one embodiment.As shown in figure 5, subject detection model Network structure include convolutional layer 402, pond layer 404, convolutional layer 406, pond layer 408, convolutional layer 410, pond layer 412, volume Lamination 414, pond layer 416, convolutional layer 418, convolutional layer 420, bilinear interpolation layer 422, convolutional layer 424, bilinear interpolation layer 426, convolutional layer 428, convolution feature articulamentum 430, bilinear interpolation layer 432, convolutional layer 434, convolution feature articulamentum 436, Bilinear interpolation layer 438, convolutional layer 440, convolution feature articulamentum 442 etc., input of the convolutional layer 402 as subject detection model Layer, output layer of the convolution feature articulamentum 442 as subject detection model.The network knot of subject detection model in the present embodiment Structure is merely illustrative, not as the limitation to the application.It is understood that the convolution in the network structure of subject detection model It is multiple that layer, pond layer, bilinear interpolation layer, convolution feature articulamentum etc. can according to need setting.

The coded portion of the subject detection model include convolutional layer 402, pond layer 404, convolutional layer 406, pond layer 408, Convolutional layer 410, pond layer 412, convolutional layer 414, pond layer 416, convolutional layer 418, decoded portion include convolutional layer 420, two-wire Property interpolated layer 422, convolutional layer 424, bilinear interpolation layer 426, convolutional layer 428, convolution feature articulamentum 430, bilinear interpolation Layer 432, convolutional layer 434, convolution feature articulamentum 436, bilinear interpolation layer 438, convolutional layer 440, convolution feature articulamentum 442.Convolutional layer 406 and convolutional layer 434 cascade (Concatenation), and convolutional layer 410 and convolutional layer 428 cascade, convolutional layer 414 cascade with convolutional layer 424.Bilinear interpolation layer 422 and convolution feature articulamentum 430 are superimposed using deconvolution feature (Deconvolution+add) it bridges.Bilinear interpolation layer 432 and convolution feature articulamentum 436 are superimposed using deconvolution feature Bridge joint.Bilinear interpolation layer 438 and convolution feature articulamentum 442 are using deconvolution feature superposition bridge joint.

Original image 450 (such as visible light figure) is input to the convolutional layer 402 of subject detection model, and depth map 460 acts on main body The convolution feature articulamentum 442 of detection model, center weight Figure 47 0 act on the convolution feature articulamentum of subject detection model 442.Depth map 460 and center weight Figure 47 0 are input to convolution feature articulamentum 442 respectively as a Product-factor.Original image 450, depth map 460 and center weight Figure 47 0 are input to the confidence level Figure 48 0 of output comprising main body after subject detection model.

The Loss Rate of default value is used in the training process of the subject detection model to depth map.The default value can be 50%.Dropout that probability is introduced in the training process of depth map allows subject detection model can sufficient excavating depth figure Information still can export accurate result when subject detection model can not obtain depth map.Depth map is inputted and is used The mode of dropout makes subject detection model more preferable to the robustness of depth map, can also accurately divide even if without depth map Body region.

In addition, the shooting and all relatively time consuming effort of calculating of depth map are difficult because in normal terminal device shooting process To obtain, in training, depth map is designed as 50% dropout probability, can guarantee that main body is examined when no depth information Surveying model still can normally detect.

Highlight detection is carried out using highlight detection layer 444 to original image 450 and identifies the highlight area in original image.Main body is examined The body region confidence level figure for surveying model output carries out adaptive threshold filtration treatment and obtains the exposure mask figure of binaryzation, to binaryzation Exposure mask figure carries out Morphological scale-space and guiding filtering handles to obtain main body exposure mask figure, by main body exposure mask figure and includes highlight area Original image carries out difference processing, and highlight area is deleted from main body exposure mask figure and obtains the main body of removal bloom.Body region confidence Degree figure is distributed across 0 to 1 confidence level figure, and the noise that body region confidence level figure includes is more, has many confidence levels lower Noise, or the fritter high confidence level region to condense together, are filtered processing by region adaptivity confidence threshold value, obtain Binaryzation exposure mask figure.Noise can be further decreased by doing Morphological scale-space to binaryzation exposure mask figure, do guiding filtering processing, can be with Make edge smoother.It is understood that body region confidence level figure can be the main body exposure mask figure comprising noise.

Use depth map as feature to enhance network output as a result, there is no directly input depth map in the present embodiment Into the network of subject detection model, a dual-depth learning network structure, one of deep learning net can be in addition designed Network structure is for handling depth map, another deep learning network structure is for handling RGB figure, then by two The output of a deep learning network structure carries out the connection of convolution feature, then exports again.

In one embodiment, the training method of the subject detection model, comprising: obtain Same Scene visible light figure and The main body exposure mask figure marked；Generate center weight figure corresponding with the visible light figure, wherein represented by the center weight figure Weighted value is gradually reduced from center to edge；The visible light figure is acted on into the subject detection model comprising initial network weight The center weight figure is acted on the output layer of initial subject detection model by input layer, the main body exposure mask figure that this has been marked As the true value of subject detection model output, which is trained, is obtained To the target network weight of the subject detection model.

Training in the present embodiment uses visible light figure and center weight figure, the i.e. network in the subject detection model of Fig. 5 Output layer part does not introduce depth map in structure, acts on convolutional layer 402 using visible light figure, center weight Figure 47 0 is acted on The convolution feature articulamentum 442 of subject detection model.

Fig. 6 is the effect diagram of target subject identification in one embodiment.As shown in fig. 6, in RGB Figure 50 2, there are one Butterfly, RGB figure is input to after subject detection model and obtains body region confidence level Figure 50 4, then to body region confidence Degree Figure 60 4 is filtered and obtains binaryzation exposure mask Figure 50 6 with binaryzation, then carries out Morphological scale-space to binaryzation exposure mask Figure 50 6 Edge enhancing is realized with guiding filtering, obtains main body exposure mask Figure 50 8.

In one embodiment, described the step of obtaining visible light figure is returned to obtain the visible light figure of different acquisition angle The step of include: the surrounding target main body continuous transformation acquisition angles, between adjacent visible light figure centered on target subject There are under conditions of overlapping region, visible light figure is acquired in real time, obtains the visible light figure of different acquisition angle.

Specifically, the transformation of acquisition angles can be realized by rotary taking device, the amplitude of transformation can customize, and such as shoot One cube rotates 45 degree of visible light figures to acquire different angle every time.Since the visible light figure needs of acquisition cover Entire target subject is covered, needs to guarantee there are overlapping region the integrality of information collection between adjacent visible light figure, wherein The ratio of overlapping can customize.

In the present embodiment, by surrounding target main body continuous transformation acquisition angles, acquiring visible light figure in real time can guarantee letter The integrality of acquisition is ceased, three-dimensionalreconstruction is carried out to target subject in real time to realize.

In one embodiment, step 210 includes: corresponding first depth of the first image plane vegetarian refreshments obtained on target subject Angle value obtains the first image plane vegetarian refreshments in three-dimensional space in the position of target subject and the first depth value according to the first image plane vegetarian refreshments Between corresponding first three-dimensional image vegetarian refreshments, obtain target subject on corresponding second depth value of the second image plane vegetarian refreshments, with first Depth value is reference depth value, determines the second three-dimensional image vegetarian refreshments relative to the first three-dimensional image vegetarian refreshments according to the second depth value Relative position, the second three-dimensional image vegetarian refreshments is the second image plane vegetarian refreshments in the corresponding three-dimensional image vegetarian refreshments of three-dimensional space, according to opposite position It sets and determines the second three-dimensional image vegetarian refreshments in the corresponding position of three-dimensional space in the position of the target subject with the second image plane vegetarian refreshments； Connect each three-dimensional image vegetarian refreshments in three-dimensional space.

Specifically, the matching of voxel point can be carried out to each planar pixel point on target subject, it can also be to target Crucial planar pixel point in main body carries out the matching of voxel point, to improve the efficiency of three-dimensionalreconstruction, saves computer money Source.In one embodiment, when target subject is face, crucial image plane vegetarian refreshments is the feature by obtaining to Face datection Key point, as selected to obtain crucial image plane vegetarian refreshments on nose, eyes, mouth, eyebrow.Implement three-dimensional model needs relative depth Value, selects a point, such as the first image plane vegetarian refreshments, obtains calibration point as basic, other points, which are calculated, puts opposite depth value with this, So as to gradually build up target subject corresponding each three-dimensional image vegetarian refreshments in three dimensions, connect each in three-dimensional space Three-dimensional image vegetarian refreshments must arrive the corresponding threedimensional model of target subject.

In one embodiment, step 210 comprises determining that the corresponding target type of target subject, is obtained according to target type The initial threedimensional model of same type is taken, the corresponding actual depth value of crucial image plane vegetarian refreshments on target subject is obtained；From first It is obtained in beginning threedimensional model and the matched threedimensional model pixel of crucial image plane vegetarian refreshments；According to each crucial image plane vegetarian refreshments it Between actual depth value ratio adjust the three-dimensional space position of matched threedimensional model pixel；Obtain the non-pass on target subject The corresponding actual depth value of key image plane vegetarian refreshments obtains and the non-key matched three-dimensional of image plane vegetarian refreshments from initial threedimensional model Model pixel point adjusts and non-pass according to the actual depth value ratio between non-key image plane vegetarian refreshments and crucial image plane vegetarian refreshments The three-dimensional space position of the matched threedimensional model pixel of key image plane vegetarian refreshments, until each image plane vegetarian refreshments on target subject There are matched threedimensional model pixels adjusted；It is corresponding that each threedimensional model pixel adjusted forms target subject Threedimensional model.

Specifically, subject detection model not only exports the target subject profile in visible light figure, also output target subject pair The target type answered, target type include people, flower, cat, dog etc..The corresponding initial three-dimensional mould of each target type can be pre-established Type, as at the beginning of establishing the initial threedimensional model of face, the initial threedimensional model of human body, spend the initial threedimensional model of initial threedimensional model, cat, dog Beginning threedimensional model etc..Crucial image plane vegetarian refreshments is the key feature points for determining that threedimensional model is three-dimensional, such as face, by right Face datection obtains crucial image plane vegetarian refreshments, such as selecting as crucial image plane vegetarian refreshments on nose, eyes, mouth, eyebrow.According to pass Actual depth value ratio between key image plane vegetarian refreshments adjusts the three-dimensional space position of matched threedimensional model pixel.To right Initial threedimensional model is adjusted to obtain and the matched threedimensional model stereo profile of target subject, such as nose of initial threedimensional model It is lower, and the nose of realistic objective main body is higher, by the way that the nose areas in initial threedimensional model is turned up to obtain and target master The matched threedimensional model stereo profile of body.It is further adjusted initially further according to the actual depth value of other non-key image plane vegetarian refreshments Threedimensional model obtains and the more matched threedimensional model of target subject.

In the present embodiment, since initial threedimensional model is to meet conventional model, depth map can be reduced due to precision and calculated Inaccurate problem, the black hole formed on depth map influence caused by the construction of threedimensional model.In the base of initial threedimensional model Gradually amendment obtains and the matched threedimensional model of target subject on plinth.

In one embodiment, it is seen that light figure acquires in real time, method further include: according to the visible light acquired in real time Figure, in the construction process of the corresponding threedimensional model of preview interface real-time display target subject.

Specifically, the visible light figure including target subject can be acquired in real time by terminal device, while acquisition gradually Construction threedimensional model corresponding with target subject, and in building in the preview interface corresponding threedimensional model of real-time display target subject Structure process, so that user can intuitively watch the construction process of threedimensional model.So as to according to the three-dimensional mould of the construction of display Type, adjust visible light figure acquisition angles so that the building of the corresponding threedimensional model of target subject it is more acurrate with it is efficient.

In one embodiment, according to the texture and color of visible light figure, the corresponding threedimensional model of target subject is matched Corresponding texture and color.Although should be understood that Fig. 2, Fig. 4 flow chart in each step according to arrow instruction according to Secondary display, but these steps are not that the inevitable sequence according to arrow instruction successively executes.Unless having herein explicitly Bright, there is no stringent sequences to limit for the execution of these steps, these steps can execute in other order.Moreover, Fig. 2, At least part step in Fig. 4 may include that perhaps these sub-steps of multiple stages or stage be not necessarily for multiple sub-steps It is so to execute completion in synchronization, but can execute at different times, these sub-steps or stage execute sequence Also it is not necessarily and successively carries out, but can be at least part of the sub-step or stage of other steps or other steps It executes in turn or alternately.

Fig. 7 is the structural block diagram of the constructing devices of threedimensional model in one embodiment.As shown in fig. 7, a kind of threedimensional model Constructing devices, including processing module 602, detection module 604, target subject determining module 606 and threedimensional model building block 608.Wherein:

Processing module 602 generates center weight figure corresponding with the visible light figure for obtaining visible light figure, wherein Weighted value represented by the center weight figure is gradually reduced from center to edge.

Detection module 604 is obtained for the visible light figure and the center weight figure to be input in subject detection model To body region confidence level figure, wherein the subject detection model is previously according to the visible light figure of Same Scene, center weight The model that figure and the corresponding main body exposure mask figure marked are trained.

Target subject determining module 606, for being determined in the visible light figure according to the body region confidence level figure Target subject.

Threedimensional model building block 608, for obtaining the corresponding depth information of the target subject, according to the target master Body and the corresponding depth information of target subject carry out three-dimensionalreconstruction to the target subject, return to the processing module to obtain The visible light figure of different acquisition angle, until obtaining the corresponding threedimensional model of the target subject.

The constructing devices of threedimensional model in the present embodiment can allow the object of picture centre more to hold using center weight figure It is easily detected, utilizes the trained subject detection obtained using the training such as visible light figure, center weight figure and main body exposure mask figure Model can more accurately identify that the target subject in visible light figure passes through target subject in threedimensional model construction Depth information realizes the accurate building of the corresponding threedimensional model of target subject, also can be accurate there are interfering object Target subject is identified, to improve the accuracy rate of the corresponding threedimensional model construction of target subject.

In one embodiment, target subject determining module 606 is also used to handle the body region confidence level figure, Obtain main body exposure mask figure；The visible light figure is detected, determines the highlight area in the visible light figure；According to the height in the visible light figure Light region and the main body exposure mask figure determine the target subject that bloom is eliminated in the visible light figure.

In one embodiment, target subject determining module 606 is also used to carry out the body region confidence level figure adaptive Confidence threshold value filtration treatment is answered, main body exposure mask figure is obtained.

In one embodiment, target subject determining module 606 is also used to carry out the body region confidence level figure adaptive Confidence threshold value filtration treatment is answered, binaryzation exposure mask figure is obtained；Morphological scale-space and guidance filter are carried out to the binaryzation exposure mask figure Wave processing, obtains main body exposure mask figure.

In one embodiment, target subject determining module 606 is also used to the highlight area in the visible light figure and is somebody's turn to do Main body exposure mask figure does difference processing, obtains the target subject in the visible light figure.

In one embodiment, which includes the input layer being sequentially connected, middle layer and output layer；

Detection module 604 is also used to act in the visible light figure input layer of subject detection model；By the center Weight map acts on the output layer of the subject detection model.

In one embodiment, threedimensional model building block 608 is also used to obtain depth corresponding with the visible light figure Figure；The depth map includes at least one of TOF depth map, binocular depth figure and structure light depth map；To the visible light figure and Depth map carries out registration process, visible light figure and depth map after being registrated, according to target subject institute in the visible light figure Region the corresponding depth information of target subject is determined from the depth map after registration.

In one embodiment, threedimensional model building block 608 is also used to centered on target subject, surrounding target main body Continuous transformation acquisition angles；There are under conditions of overlapping region between adjacent visible light figure, visible light figure is acquired in real time, is obtained To the visible light figure of different acquisition angle.

In one embodiment, threedimensional model building block 608 is also used to obtain the first planar pixel on target subject Corresponding first depth value of point；The first plane is obtained in the position of target subject and the first depth value according to the first image plane vegetarian refreshments Pixel is in the corresponding first three-dimensional image vegetarian refreshments of three-dimensional space；Obtain the second image plane vegetarian refreshments corresponding second on target subject Depth value；Using the first depth value as reference depth value, determine the second three-dimensional image vegetarian refreshments relative to described according to the second depth value The relative position of one three-dimensional image vegetarian refreshments, the second three-dimensional image vegetarian refreshments are the second image plane vegetarian refreshments in the corresponding voxel of three-dimensional space Point；Determine the second three-dimensional image vegetarian refreshments in three-dimensional space pair in the position of target subject with the second image plane vegetarian refreshments depending on the relative position The position answered；Connect each three-dimensional image vegetarian refreshments in three-dimensional space.

In one embodiment, threedimensional model building block 608 is also used to determine the corresponding target type of target subject, root The initial threedimensional model of same type is obtained according to target type；Obtain the corresponding reality of crucial image plane vegetarian refreshments on target subject Depth value；Acquisition and the matched threedimensional model pixel of the crucial image plane vegetarian refreshments from initial threedimensional model, according to each Actual depth value ratio between crucial image plane vegetarian refreshments adjusts the three-dimensional space position of matched threedimensional model pixel；It obtains The corresponding actual depth value of non-key image plane vegetarian refreshments on target subject；From initial threedimensional model obtain with it is described non-key The matched threedimensional model pixel of image plane vegetarian refreshments；According to the reality between non-key image plane vegetarian refreshments and crucial image plane vegetarian refreshments The three-dimensional space position of depth value ratio adjustment and the matched threedimensional model pixel of non-key image plane vegetarian refreshments, until target master There are matched threedimensional model pixels adjusted for each image plane vegetarian refreshments on body；Each threedimensional model pixel adjusted Point forms the corresponding threedimensional model of target subject.

In one embodiment, device further include:

Display module, it is corresponding in preview interface real-time display target subject for the visible light figure that basis acquires in real time The construction process of threedimensional model.

In one embodiment, detection module 604 is also used to visible light figure, depth map and the center weight after the registration Figure is input in subject detection model, obtains body region confidence level figure；Wherein, which is previously according to same The model that visible light figure, depth map, center weight figure and the corresponding main body exposure mask figure marked of scene are trained.

In one embodiment, the constructing devices of above-mentioned threedimensional model further include that training image obtains module, training weight Generation module and training module.

Training image obtains the visible light figure, depth map and the main body exposure mask marked that module is used to obtain Same Scene Figure.

Training weight generation module is for generating center weight figure corresponding with the visible light figure, wherein the center Weighted value represented by weight map is gradually reduced from center to edge.

Training module is used to act in the visible light figure input of the subject detection model comprising initial network weight The depth map and the center weight figure, are acted on the output layer of initial subject detection model, marked described by layer The true value that is exported as the subject detection model of main body exposure mask figure, to the subject detection comprising initial network weight Model is trained, and obtains the target network weight of the subject detection model.When the loss function of subject detection model is less than When loss threshold value or frequency of training reach preset times, the network weight of subject detection model is as subject detection model Target network weight.

Fig. 8 is the schematic diagram of internal structure of terminal device in one embodiment.As shown in figure 8, the terminal device includes logical Cross the processor and memory of system bus connection.Wherein, which supports entirely eventually for providing calculating and control ability The operation of end equipment.Memory may include non-volatile memory medium and built-in storage.Non-volatile memory medium is stored with behaviour Make system and computer program.The computer program can be performed by processor, for realizing provided by each embodiment A kind of constructing method of threedimensional model.Built-in storage provides height for the operating system computer program in non-volatile memory medium The running environment of speed caching.The terminal device can be mobile phone, tablet computer or personal digital assistant or wearable device etc..

The realization of modules in the constructing devices of the threedimensional model provided in the embodiment of the present application can be computer journey The form of sequence.The computer program can be run in terminal or server.The program module that the computer program is constituted can store On the memory of terminal or server.When the computer program is executed by processor, realize described in the embodiment of the present application The step of method.

The embodiment of the present application also provides a kind of computer readable storage mediums.One or more is executable comprising computer The non-volatile computer readable storage medium storing program for executing of instruction, when the computer executable instructions are executed by one or more processors When, so that the step of processor executes the constructing method of threedimensional model.

A kind of computer program product comprising instruction, when run on a computer, so that computer executes three-dimensional The step of constructing method of model.

It may include non-to any reference of memory, storage, database or other media used in the embodiment of the present application Volatibility and/or volatile memory.Suitable nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include Random access memory (RAM), it is used as external cache.By way of illustration and not limitation, RAM in a variety of forms may be used , such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM).

The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously The limitation to the application the scope of the patents therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art For, without departing from the concept of this application, various modifications and improvements can be made, these belong to the guarantor of the application Protect range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.

Claims (10)

1. a kind of constructing method of threedimensional model, which is characterized in that the described method includes:

Visible light figure is obtained, generates center weight figure corresponding with the visible light figure, wherein represented by the center weight figure Weighted value be gradually reduced from center to edge；

The visible light figure and the center weight figure are input in subject detection model, body region confidence level figure is obtained, Wherein, the subject detection model is the visible light figure previously according to Same Scene, center weight figure and corresponding has marked The model that main body exposure mask figure is trained；

The target subject in the visible light figure is determined according to the body region confidence level figure；

Obtain the corresponding depth information of the target subject；

According to the target subject and the corresponding depth information of target subject, three-dimensionalreconstruction is carried out to the target subject, is returned Described the step of obtaining visible light figure, is corresponding until obtaining the target subject to obtain the visible light figure of different acquisition angle Threedimensional model.

2. the method according to claim 1, wherein described according to body region confidence level figure determination Target subject in visible light figure, comprising:

The body region confidence level figure is handled, main body exposure mask figure is obtained；

The visible light figure is detected, determines the highlight area in the visible light figure；

According in the visible light figure highlight area and the main body exposure mask figure, determine and eliminate bloom in the visible light figure Target subject.

3. the method according to claim 1, wherein described obtain the corresponding depth information packet of the target subject It includes:

Obtain depth map corresponding with the visible light figure；The depth map includes TOF depth map, binocular depth figure and structure light At least one of depth map；

Registration process, visible light figure and depth map after being registrated are carried out to the visible light figure and depth map；

Region according to where target subject described in the visible light figure determines the target master from the depth map after registration The corresponding depth information of body.

4. the method according to claim 1, wherein the step of return acquisition visible light figure, is to obtain The step of visible light figure of different acquisition angle includes:

Centered on the target subject, the target subject continuous transformation acquisition angles are surrounded；

There are under conditions of overlapping region between adjacent visible light figure, visible light figure is acquired in real time, the difference is obtained and adopts Collect the visible light figure of angle.

5. the method according to claim 1, wherein described corresponding according to the target subject and target subject Depth information carries out three-dimensionalreconstruction to the target subject, comprising:

Obtain corresponding first depth value of the first image plane vegetarian refreshments on the target subject；

It is flat that in the position of the target subject and first depth value described first is obtained according to the first image plane vegetarian refreshments Image surface vegetarian refreshments is in the corresponding first three-dimensional image vegetarian refreshments of three-dimensional space；

Obtain corresponding second depth value of the second image plane vegetarian refreshments on the target subject；

Using first depth value as reference depth value, determine the second three-dimensional image vegetarian refreshments relative to institute according to second depth value The relative position of the first three-dimensional image vegetarian refreshments is stated, the second three-dimensional image vegetarian refreshments is that the second image plane vegetarian refreshments is corresponding in three-dimensional space Three-dimensional image vegetarian refreshments；

Determine that described second is three-dimensional in the position of the target subject with the second image plane vegetarian refreshments depending on that relative position Pixel is in the corresponding position of the three-dimensional space；

Connect each three-dimensional image vegetarian refreshments in the three-dimensional space.

6. the method according to any one of claims 1 to 5, which is characterized in that described according to the target subject and mesh The step of marking the corresponding depth information of main body, carried out by three-dimensionalreconstruction, returns to the acquisition visible light figure for the target subject with Obtain different acquisition angle visible light figure, include: until obtaining the corresponding threedimensional model of the target subject

It determines the corresponding target type of target subject, the initial threedimensional model of same type is obtained according to the target type；

Obtain the corresponding actual depth value of crucial image plane vegetarian refreshments on the target subject；

Acquisition and the matched threedimensional model pixel of the crucial image plane vegetarian refreshments from the initial threedimensional model, according to each Actual depth value ratio between crucial image plane vegetarian refreshments adjusts the three-dimensional space position of matched threedimensional model pixel；

Obtain the corresponding actual depth value of non-key image plane vegetarian refreshments on the target subject；

It is obtained and the non-key matched threedimensional model pixel of image plane vegetarian refreshments from the initial threedimensional model；

According to the adjustment of actual depth value ratio and non-key plane between non-key image plane vegetarian refreshments and crucial image plane vegetarian refreshments The three-dimensional space position of the matched threedimensional model pixel of pixel, until each image plane vegetarian refreshments presence on target subject The threedimensional model pixel adjusted matched；

Each threedimensional model pixel adjusted forms the corresponding threedimensional model of target subject.

7. the method according to any one of claims 1 to 5, which is characterized in that the visible light figure acquires in real time, The method also includes:

According to the visible light figure acquired in real time, the corresponding threedimensional model of the target subject described in preview interface real-time display Construction process.

8. a kind of constructing devices of threedimensional model, which is characterized in that described device includes:

Processing module generates center weight figure corresponding with the visible light figure, wherein in described for obtaining visible light figure Weighted value represented by heart weight map is gradually reduced from center to edge；

Detection module obtains main body for the visible light figure and the center weight figure to be input in subject detection model Region confidence figure, wherein the subject detection model is the visible light figure previously according to Same Scene, center weight figure and right The model that the main body exposure mask figure marked answered is trained；

Target subject determining module, for determining the target master in the visible light figure according to the body region confidence level figure Body；

Threedimensional model building block, for obtaining the corresponding depth information of the target subject, according to the target subject and mesh The corresponding depth information of main body is marked, three-dimensionalreconstruction is carried out to the target subject, the processing module is returned and is adopted with obtaining difference The visible light figure for collecting angle, until obtaining the corresponding threedimensional model of the target subject.

9. a kind of terminal device, including memory and processor, computer program, the computer are stored in the memory When program is executed by the processor, so that the processor executes the step of the method as described in any one of claims 1 to 7 Suddenly.

10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program The step of method as described in any one of claims 1 to 7 is realized when being executed by processor.