CN104966284A - Method and equipment for acquiring object dimension information based on depth data - Google Patents
Method and equipment for acquiring object dimension information based on depth data Download PDFInfo
- Publication number
- CN104966284A CN104966284A CN201510290441.4A CN201510290441A CN104966284A CN 104966284 A CN104966284 A CN 104966284A CN 201510290441 A CN201510290441 A CN 201510290441A CN 104966284 A CN104966284 A CN 104966284A
- Authority
- CN
- China
- Prior art keywords
- depth image
- special object
- key point
- information
- specific key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Abstract
The invention provides a method and a piece of equipment for acquiring object dimension information based on depth data. The method acquiring object dimension information based on depth data comprises the following steps of positioning multiple specific key points of a specific object in a depth image; generating the spatial coordinates of the specific key points of the specific object; and calculating the dimension information of the specific object on the basis of the spatial coordinates of the specific key points of the specific object. When the specific object is a human body, a clothes size recommendation of the human body can be provided according to the figure information of the human body. The method and the equipment have great application prospects in terms of on-line clothes buying or physical-store clothes buying.
Description
Technical field
The present invention relates to three-dimensional information extraction technical field, relate more specifically to a kind of method and apparatus obtaining object size information based on depth data.
Background technology
Universal along with ecommerce, has become at online purchase clothes the shopping way that numerous network user makes earnest efforts.But to buy clothes different from solid shop, and the network user cannot try on a dress when online purchase clothes, and can only estimate the size of clothes that oneself may be suitable, this also often result in the inappropriate problem of suit length that the network user has bought on the net.When solid shop buys clothes, user needs the size being determined clothes needed for oneself by the clothes trying different size on, and this is also often consuming time.
Therefore, no matter be buy clothes for online purchase clothes or at solid shop, all need a kind of technology providing the clothes recommendation being suitable for this user according to the actual stature of user in three dimensional physical world space.
Summary of the invention
Consider the problems referred to above and propose the present invention.The invention provides a kind of method and apparatus obtaining object size information based on depth data, by extracting the specific key point of special object and determine its three-dimensional coordinate from depth image, the Three Dimensions Structure of described special object can be determined.
According to an aspect of the present invention, provide a kind of method obtaining object size information based on depth data, comprising: the multiple specific key point of locating special object in depth image; Produce the volume coordinate of the multiple specific key point of described special object; And the volume coordinate of multiple specific key point based on described special object, calculate the dimension information of described special object.
According to the disclosure on the other hand, additionally providing a kind of equipment obtaining object size information based on depth data, comprising: depth image harvester, for taking the depth image of special object; One or more processor; One or more storer; And the computer program instructions stored in which memory, perform following steps when described computer program instructions is run by described processor: the multiple specific key point of locating described special object in described depth image; Produce the volume coordinate of the multiple specific key point of described special object; And the volume coordinate of multiple specific key point based on described special object, calculate the dimension information of described special object.
According to the another aspect of the disclosure, providing a kind of equipment obtaining object size information based on depth data, comprising: depth image harvester, for taking the depth image of special object; Key point locating device, for locating the multiple specific key point of described special object in described depth image; Coordinate transformation device, for generation of the volume coordinate of the multiple specific key point of described special object; And Size calculation device, for the multiple specific key point based on described special object spatial coordinates calculation described in the dimension information of special object.
According to the disclosure more on the one hand, provide a kind of computer program for obtaining object size information based on depth data, comprise one or more computer-readable recording medium, described computer-readable recording medium stores computer program instructions, and described computer program instructions can be performed to make described processor by processor: the multiple specific key point of locating special object in depth image; Produce the volume coordinate of the multiple specific key point of described special object; And the volume coordinate of multiple specific key point based on described special object, calculate the dimension information of described special object.
According to disclosure embodiment, from depth image, the specific key point of special object is extracted according to key point geometric model, then the three-dimensional coordinate of described specific key point under physical world coordinates system is determined, thus the critical size information of described special object can be extracted exactly, and obtain the key structural features of described special object.Advantageously, the subsequent operations such as dimension inspection, dimension correction, size recommendation can also be carried out based on extracted critical size information.More advantageously, when described special object is human body, can provide the suit length recommendation of human body according to the body information of human body, this buys clothes for online purchase clothes or solid shop and all has a good application prospect.
Accompanying drawing explanation
Be described in more detail disclosure embodiment in conjunction with the drawings, above-mentioned and other object of the present invention, Characteristics and advantages will become more obvious.Accompanying drawing is used to provide the further understanding to disclosure embodiment, and forms a part for instructions, is used from explanation the present invention, is not construed as limiting the invention with disclosure embodiment one.In the accompanying drawings, identical reference number represents same parts or step usually.
Fig. 1 is the schematic block diagram obtaining the example electronic device of the method and apparatus of object size information based on depth data illustrated for realizing disclosure embodiment;
Fig. 2 is the indicative flowchart of the method based on depth data acquisition object size information according to disclosure embodiment;
Fig. 3 is the schematic diagram of the specific key point of human body according to disclosure embodiment; And
Fig. 4 is the schematic block diagram of the equipment based on depth data acquisition object size information according to disclosure embodiment.
Embodiment
In order to make the object, technical solutions and advantages of the present invention more obvious, describe in detail below with reference to accompanying drawings according to example embodiment of the present invention.Obviously, described embodiment is only a part of embodiment of the present invention, instead of whole embodiment of the present invention, should be understood that the present invention not by the restriction of example embodiment described herein.Based on the disclosure embodiment described in the disclosure, other embodiments all that those skilled in the art obtain when not paying creative work all should fall within protection scope of the present invention.
Depth image harvester can provide the depth information of photographed scene in real world, according to the depth information that depth camera gathers, can realize the reconstruction in the three dimensional physical world, namely can various subjects in the three dimensional physical world of rediscover.Described depth image harvester can be such as depth camera, video camera etc.At present, depth camera obtains widespread use gradually, and is mainly used in field of human-computer interaction, the throw-in type game etc. of such as gesture identification and family.But depth camera is seldom used to other applied environment.
According to disclosure embodiment, utilize depth image harvester to gather the depth data of special object in the three dimensional physical world, the depth data collected according to depth image harvester extracts the dimension information of described special object.
First, the example electronic device 100 of the method and apparatus of the object size information that obtains based on depth data for realizing disclosure embodiment is described with reference to Fig. 1.
As shown in Figure 1, electronic equipment 100 comprises one or more processor 102, one or more memory storage 104, input media 106, output unit 108 and depth image harvester 110, and these assemblies are interconnected by bindiny mechanism's (not shown) of bus system 112 and/or other form.The assembly and the structure that it should be noted that the electronic equipment 100 shown in Fig. 1 are illustrative, and not restrictive, and as required, described electronic equipment also can have other assemblies and structure.
Described processor 102 can be the processing unit of CPU (central processing unit) (CPU) or other form with data-handling capacity and/or instruction execution capability, and other assembly that can control in described electronic equipment 100 is with the function of carry out desired.
Described memory storage 104 can comprise one or more computer program, and described computer program can comprise various forms of computer-readable recording medium, such as volatile memory and/or nonvolatile memory.Described volatile memory such as can comprise random access memory (RAM) and/or cache memory (cache) etc.Described nonvolatile memory such as can comprise ROM (read-only memory) (ROM), hard disk, flash memory etc.Described computer-readable recording medium can store one or more computer program instructions, processor 102 can run described programmed instruction, to realize function and/or other function expected of (being realized by processor) in disclosure embodiment hereinafter described.Various application program and various data can also be stored, the various data etc. that such as depth image, key point model space geometric and described application program use and/or produce in described computer-readable recording medium.
Described input media 106 can be that user is used for inputting the device of instruction, and it is one or more to comprise in keyboard, mouse, microphone and touch-screen etc.Described instruction is such as the instruction using described depth image harvester 110 to take depth image, or uses described electronic equipment 100 to carry out the instruction of suit length recommendation.
Described output unit 108 externally (such as user) can export various information (such as image or sound), and it is one or more to comprise in display, loudspeaker etc.
Described depth image harvester 110 can take the image (such as photo, video etc.) that user expects, and is stored in described memory storage 104 by captured image and uses for other assembly.
Described depth image harvester 110 can be arranged together with described processor 102 and described memory storage 104, or can arrange discretely with described processor 102 and described memory storage 104.
Preferably, the example electronic device 100 for the method and apparatus based on depth data acquisition object size information realizing disclosure embodiment may be implemented as the mobile terminals such as such as smart mobile phone, panel computer, wearable device.But, the disclosure is not limited thereto, assembly at least partially in described electronic equipment 100 can be fixing, such as described depth image harvester 110 can be fixed on predetermined altitude, described processor 102, memory storage 104, input media 106 and output unit 108 can be mounted together and can as independently fixed terminal or mobile terminal, described fixed terminal can comprise the electronic equipment of the usual not shift positions such as desk-top computer, can be communicated by wired or wireless mode with described independently fixed terminal or mobile terminal at described depth image harvester 110, in the case, described electronic equipment 100 can also comprise the communicator (not shown) for realizing the wired or wireless communication between described depth image harvester 110 and described independently fixed terminal or mobile terminal.In addition, according to disclosure embodiment, described electronic equipment 100 also can receive depth image from the depth image harvester be separated, in the case, electronic equipment 100 also can not comprise described depth image harvester, and can comprise the wired or wireless communication device (not shown) for receiving depth image from the depth image harvester be separated.
Below, the method for the object size information that obtains based on depth data according to disclosure embodiment is described with reference to Fig. 2.
As shown in Figure 2, in step S210, the depth image of shooting special object.Particularly, the depth image harvester 110 in the electronic equipment 100 of the method based on depth data acquisition object size information for realizing disclosure embodiment as shown in Figure 1 or other depth image harvester that can transmit depth image to described electronic equipment 100 independent of described electronic equipment 100 can be utilized.Described depth image can be photo, also can be the frame in video.Described photo can comprise the photo of the single scene of one or more, also can be distant view photograph.
In step S220, in described depth image, locate the multiple specific key point of special object.Described special object can be determined according to actual application environment, such as under the applied environment of human body stature data extraction, described special object can be human body, under the applied environment that the figure data of particular types animal are extracted, described special object can be this species, and under the applied environment extracted at the dimensional data of certain article, described special object can be these article.For the special object of each kind, described multiple specific key point can be different.Such as, described specific key point can be the key point of the architectural feature of described special object, can be the extreme point in the appearance profile of described special object or flex point particularly, described in described extreme point place, a certain geometric parameter of the appearance profile of special object has maximum value or minimum value, and described in described flex point place, the variable quantity trend of a certain geometric parameter of the appearance profile of special object changes.
Alternatively, locate the multiple specific key point of special object in described depth image before, pre-service can be carried out to described depth image, be beneficial to the location of described specific key point.Such as, when described image is photo, described pre-service can comprise comparison film and carry out convergent-divergent and/or noise reduction process, and when described image is video, described pre-service can comprise the key frame etc. extracting video.
Particularly, according to disclosure embodiment, first can orient described special object in depth image, and then on described special object, locate described multiple specific key point.
Alternatively, background modeling algorithm can be adopted, according to the background in background information determination current depth image, then determine the prospect in current depth image, and using the prospect in determined current depth image as described special object.Such as, relative to shooting background, the depth image harvester for taking described depth image can be installed in advance, described shooting background can be such as wall or background cloth etc., described depth image harvester such as can be arranged on the position apart from 1.5 meters to 2 meters, ground, namely be 1.5 meters to 2 meters relative to the setting height(from bottom) on ground, its visual angle can be look squarely.When being positioned at before described shooting background without any object, described depth image harvester can be utilized to take the depth image of described shooting background (hereinafter referred to background depth image) in advance, thus the background depth information of described shooting background can be obtained.When described special object is positioned at before described shooting background, described depth image harvester is utilized to take the depth image (hereinafter referred to current depth image) comprising the scene of special object, with reference to described background depth image, the background parts in current depth image and prospect part can be determined.Such as, current depth image and described background depth image can be compared by pixel, for each pixel, when the depth value of both is identical, this pixel is defined as the pixel in background parts, and when the depth value of both is different, this pixel is defined as the pixel in prospect part.Alternatively, current depth image and described background depth image can be subtracted each other by pixel, for each pixel, when the absolute value of the difference of both is less than the first depth threshold (such as difference is close to zero), this pixel is defined as the pixel in background parts; And when the depth value of both is greater than described first depth threshold, this pixel is defined as the pixel in prospect, and the described depth value comprising this pixel in the depth image of the scene of special object is defined as the depth value of this pixel.
Alternatively, when described special object is human body, pedestrian detection algorithm can also be adopted, by utilizing histogram of gradients (histogram of gradient) and support vector machine (support vector machine) algorithm or training pedestrian's analytical model based on the algorithm of the model (deformable part based model) of deformable component, based on described pedestrian's analytical model, each rectangular area in current depth image is analyzed, thus find the rectangular area at pedestrian place.Then, can split according to depth information the depth image of this rectangular area, thus orient described special object in described depth image.
Orient described special object in described depth image after, next need on described special object, locate described multiple specific key point.As previously mentioned, according to variety classes or the classification of special object, described special object can have different specific key points.
According to disclosure embodiment, the multiple specific key point of pre-defined special object to be identified, and set up key point geometric model for described multiple specific key point in advance alternatively, in this key point geometric model, identify the position of each specific key point in described multiple specific key point.Here, described key point geometric model can identify each position of specific key point in the plane of delineation in described multiple specific key point, or can identify each position of specific key point in physical world three dimensions in described multiple specific key point.In described key point geometric model, the position of each specific key point can define according to the design feature of described special object.After establishing described key point geometric model, described key point geometric model can be utilized to fit within described depth image the described special object of locating, thus the particular location of each specific key point on described special object can be determined.
Such as, when described special object is human body, described specific key point can comprise the articulation point of human body.As shown in Figure 3, the example of the articulation point of human body is shown.In figure 3,15 human joint pointses such as the crown of human body, neck, chest center, left shoulder, left hand elbow, left hand, right shoulder, right hand elbow, the right hand, left stern, left knee, left foot, right stern, right knee, right crus of diaphragm are shown.
But, the described specific key point of human body is not limited only to the human joint points shown in Fig. 3, and can include but not limited to: at the bottom of the crown, neck, chest center, left shoulder, left hand elbow, left finesse, right shoulder, right hand elbow, right finesse, left stern, left knee, left ankle, left foot, at the bottom of right stern, right knee, right ankle, right crus of diaphragm.In addition, as required, the specific key point of human body can also comprise left chest edge, right chest edge, left waist edge, right waist edge, waist front center point, lumbar dorsal central point, left hip, right hip, crotch's front center point, buttocks back side central point etc.
Such as, the key point geometric model of human body can be set up for above-mentioned specific key point.Particularly, such as, for the human body of frontal upright, the peak of human body is defined as " crown " key point, the geometric center of human body is defined as " body centre (waist central point) " key point, the leftmost side point of position of being uprushed by the upper part of the body width of human body is defined as " left shoulder " key point and is defined as " right shoulder " by rightmost side point, is defined as " left hand elbow " etc. the position of downward 1/2 brachium of " left shoulder " key point.
Alternatively, when described special object is human body, for the ease of extracting each key point of described special object, when taking depth image, can require that described special object makes specific action, thus more contribute to determining the described position of multiple feature key points on described special object.Such as, described specific action can be frontal upright, the back side uprightly, with arms akimbo upright etc.
In addition, should be appreciated that, described depth image can comprise at least one depth image, such as described depth image comprises the first depth image and the second depth image, described first depth image is the direct picture of described special object, and described second depth image is back side image or the side image of described special object.
Return Fig. 2, in step S230, produce the volume coordinate of the multiple specific key point of described special object.Particularly, described volume coordinate is the intrasystem three-dimensional coordinate of physical world coordinates.Based on the imaging ratio of the height of described depth image harvester in physical world, shooting angle and described depth image acquisition component, the volume coordinate of the multiple specific key point of described special object can be produced according to described depth image.
According to disclosure embodiment, as mentioned above, described depth image harvester has predetermined altitude relative to ground, and when described special object is human body, described human body is stood on the ground, and the depth image of described human body taken by described depth image harvester.Because described depth image harvester exists certain imaging angle, such as described Depth Imaging harvester is 1.5 meters relative to the setting height(from bottom) on ground, and for the camera plane at described special object place, suppose that the horizontal range of this camera plane and described depth image harvester is 1.5 meters, described depth image harvester can gather the height from ground to 2.5 meters, the i.e. shooting height scope of 0-2.5 rice, in the case, described depth image harvester imaging angular range in the vertical direction can be-45 °-33 °.In addition, described depth image harvester imaging angular range in the horizontal direction can also be determined similarly.Therefore, need when producing the volume coordinate of multiple specific key point of described special object according to depth image to consider the imaging angular range of vertical direction and the imaging angular range of horizontal direction.
Particularly, for each specific key point in depth image, according to the horizontal range of the imaging ratio of the described position of specific key point in described depth image, described depth image harvester, described depth image harvester and described special object and the depth value at described specific key point place, determine that described specific key point is at the intrasystem three-dimensional coordinate of physical world coordinates.
Alternatively, two-dimensional coordinate system can be set up on described depth image, such as, be the initial point of described two-dimensional coordinate system with the lower left corner of described depth image, the upper left corner, the lower right corner, the upper right corner or central point, set up two-dimensional coordinate system.On the basis of this two-dimensional coordinate system, increase depth dimension, virtual three-dimensional system of coordinate can be formed.Such as, can according to the imaging ratio of described depth image harvester, the horizontal range of described depth image harvester and described special object, the setting height(from bottom) of described depth image harvester, and in other acquisition parameters of described depth image harvester at least partially, calculate the transformation matrix of coordinates between described virtual three-dimensional system of coordinate and described physical world coordinates system, thus the coordinate position of each specific key point of direct basis in described depth image and depth data can obtain the three-dimensional coordinate of this specific key point in described physical world coordinates system.Alternatively, can calculate described transformation matrix of coordinates by the method for reference field and reference point, described reference field can be the plane at described shooting background place, and described reference point can for the some calculation levels selected on described shooting background.
Alternatively, two-dimensional coordinate system can be set up on described depth image, such as, be the initial point of described two-dimensional coordinate system with the lower left corner of described depth image, the upper left corner, the lower right corner, the upper right corner or central point, set up two-dimensional coordinate system.Can centered by described depth image harvester, set up image collecting device coordinate system (referred to as camera coordinates system), this camera coordinates is three-dimensional system of coordinate, with the shooting angle of described depth image harvester for depth direction (Z-direction), and construct XY axial plane with the plane vertical with described depth direction.As previously mentioned, when the shooting angle of described depth image harvester is for looking squarely, using the angle of looking squarely as Z-direction, namely described Z-direction is the direction with plane-parallel, and constructs XY axial plane with the plane vertical with Z axis.According to the imaging ratio of described depth image harvester, described depth image harvester and the transformation matrix two-dimensional coordinate of described depth image and corresponding depth value being converted into the three-dimensional coordinate in described camera coordinates system can be determined at least partially in other acquisition parameters of the horizontal range of described special object, the setting height(from bottom) of described depth image harvester and described depth image harvester, in described camera coordinates system, the size of described special object is represented by the long measure of physical world (such as, rice, centimetre etc.).Then, can according to the direction of the XYZ axle of the direction of the XYZ axle of the setting height(from bottom) of described depth image harvester, described camera coordinates system and described physical world coordinates system, determine the coordinate conversion relation of the three-dimensional coordinate of three-dimensional coordinate in described camera coordinates system and described physical world coordinates system, such as rotational transform relation.In the case, first the three-dimensional coordinate of each specific key point under described camera coordinates system can be produced according to the view data of described depth image, then can by the three-dimensional coordinate under the three-dimensional coordinate transformation of each specific key point under described camera coordinates system to described physical world coordinates system.
In step S240, based on the volume coordinate of the multiple specific key point of described special object, calculate the dimension information of described special object.
Such as, when described special object is human body, the dimension information of described special object can be the body information of described special object, and can include but not limited to: height information, brachium information, shoulder breadth information, chest measurement information, waistline information, leg long message, upper body length information, lower-leg length information, thigh circumference information, calf circumference information etc.
Particularly, the height of human body can be determined based on the distance between the crown of human body and sole, the brachium of human body can be determined based on the distance between the left shoulder of human body (right shoulder) to left finesse (right finesse), the shoulder breadth of human body can be determined based on the distance between the left shoulder of human body and right shoulder, can based on the left chest edge of human body, distance between right chest edge and chest front center point determines the chest measurement of human body, can based on the left waist edge of human body, right waist edge and waist front center point determine the waistline of human body, can based on the left hip edge of human body, right hip edge, belly front center point and human body back side buttocks central point determine the hip circumference of human body, can determine that the leg of human body is long based on the distance at the bottom of the left hip of human body (right hip) edge and left foot between (at the bottom of right crus of diaphragm), the upper body length of human body can be determined based on the distance between the neck of human body to belly front center point, the lower-leg length of human body can be determined based on the left knee of human body (right knee) distance at the bottom of left foot between (at the bottom of right crus of diaphragm).
Next, alternatively, in step S250, based on the dimension information of described special object, size recommendation information is provided.
Such as, when described special object is special article, the size recommendation information of the package for holding described special article can be provided according to the size of described special article; When described special object is processing work, size-modify information for improvement of described processing work can be provided according to the size of described special article.
Again such as, when described special object is human body, can based on the body information of described human body, the suit length providing described human body is recommended.
Alternatively, training data can be obtained in advance, and utilize training data to carry out training classifier.Described training data can be the body information of N number of human body and corresponding suit length information.After sorter has been trained, the human body body information extracted can be inputed to described sorter from current depth image, the suit length that described sorter provides described human body based on inputted human body body information is recommended.Described suit length recommendation can be a concrete suit length, or can be an optional suit length scope.
Be shown in broken lines step S210 and S250 in fig. 2 to represent the difference between itself and step S220-S240, step S220-S240 is the necessary step of the method based on depth data acquisition object size information according to disclosure embodiment, and step S210 and S250 is then optional step.
Below, the equipment of the object size information that obtains based on depth data according to disclosure embodiment is described with reference to Fig. 4.The described equipment based on depth data acquisition object size information can to perform the above method, and can be the mobile terminal that user carries, or can be fixing electronic equipment.Each details operated performed due to the described equipment obtaining object size information based on depth data is substantially identical with the method described for Fig. 2 hereinbefore, therefore in order to avoid repeating, only concise and to the point description is carried out to the described equipment based on depth data acquisition object size information hereinafter, and omit the description to same detail.
As shown in Figure 4, depth image harvester 410, key point locating device 420, coordinate transformation device 430, Size calculation device 440, size recommendation apparatus 450, output unit 460 and memory storage 470 is comprised according to the equipment 400 based on depth data acquisition object size information of disclosure embodiment.In one example, depth image harvester 410 can realize by depth image harvester 110 as shown in Figure 1, described key point locating device 420, coordinate transformation device 430, Size calculation device 440 and size recommendation apparatus 450 can realize by processor 102 as shown in Figure 1, and described output unit 460 can realize by output unit 108 as shown in Figure 1, described memory storage 470 can realize by memory storage 104 as shown in Figure 1.In another example, depth image harvester 410 can be realized by depth camera, key point locating device 420, coordinate transformation device 430, Size calculation device 440, size recommendation apparatus 450, output unit 460, memory storage 470, output unit 108 all can be realized by the combination of hardware or hardware and software respectively, wherein, the concrete structure of each device and implementation can adopt existing various feasible scheme, do not limit at this.
Described depth image harvester 410 can the depth image of special object.Described depth image harvester 410 can be the depth image harvester on the mobile terminals such as such as smart mobile phone, panel computer, wearable device, in the case, the described equipment 400 based on depth data acquisition object size information can be implemented as described mobile terminal.Described depth image harvester 410 also can be the depth image harvester being fixed on specific location, and can with mobile terminal, desk-top computer and the notebooks etc. such as such as smart mobile phone, panel computer, wearable device in a wired fashion or wireless communication mode, in the case, the described equipment 400 based on depth data acquisition object size information can be implemented as and is fixed on the depth image harvester of specific location and the combination of mobile terminal, desk-top computer or notebook etc.
As described above, described depth image can be photo, also can be the frame in video.Described photo can comprise the photo of the single scene of one or more, also can be distant view photograph.Certainly, other capture apparatus also can be utilized to take described depth image, and the depth image of shooting be sent to the described equipment 400 obtaining object size information based on depth data, in the case, depth image harvester 410 can be omitted.
The multiple specific key point of special object located by key point locating device 420 in described depth image.Described special object can be determined according to actual application environment, such as under the applied environment of human body stature data extraction, described special object can be human body, under the applied environment that the figure data of particular types animal are extracted, described special object can be this species, and under the applied environment extracted at the dimensional data of certain article, described special object can be these article.For the special object of each kind, described multiple specific key point can be different.
Alternatively, locate the multiple specific key point of special object in described depth image before, pre-service can be carried out to described depth image, be beneficial to the location of described specific key point.Such as, when described image is photo, described pre-service can comprise comparison film and carry out convergent-divergent and/or noise reduction process, and when described image is video, described pre-service can comprise the key frame etc. extracting video.
Particularly, according to disclosure embodiment, first described key point locating device 420 can orient described special object in depth image, and then on described special object, locate described multiple specific key point.In the case, described key point locating device 420 can comprise object extraction device (not shown) and key point determining device (not shown), described object extraction device is used in depth image, orient described special object, and described key point determining device for locating described multiple specific key point on described special object.
As previously mentioned, described object extraction device can adopt background modeling algorithm, according to the background in background information determination current depth image, then determine the prospect in current depth image, and using the prospect in determined current depth image as described special object.Such as, utilize described depth image harvester when being positioned at before shooting background without any object, take the depth image (hereinafter referred to background depth image) of described shooting background in advance, thus the background depth information of described shooting background can be obtained; Then, when described special object is positioned at before described shooting background, shooting comprises the depth image (hereinafter referred to current depth image) of the scene of special object.Described object extraction device with reference to described background depth image, can determine the background parts in current depth image and prospect part.
Alternatively, when described special object is human body, described object extraction device can adopt pedestrian detection algorithm, by utilizing histogram of gradients and algorithm of support vector machine or training pedestrian's analytical model based on the algorithm of the model of deformable component, based on described pedestrian's analytical model, each rectangular area in current depth image is analyzed, thus find the rectangular area at pedestrian place.Then, can split according to depth information the depth image of this rectangular area, thus orient described special object in described depth image.
According to disclosure embodiment, the multiple specific key point of special object to be identified can be pre-defined, and set up key point geometric model for described multiple specific key point in advance alternatively, in this key point geometric model, identify the position of each specific key point in described multiple specific key point.Here, described key point geometric model can identify each position of specific key point in the plane of delineation in described multiple specific key point, or can identify each position of specific key point in physical world three dimensions in described multiple specific key point.In described key point geometric model, the position of each specific key point can define according to the design feature of described special object.Described key point determining device can utilize described key point geometric model to fit within described depth image the described special object of locating, thus can determine the particular location of each specific key point on described special object.
Alternatively, when described special object is human body, for the ease of extracting each key point of described special object, when taking depth image, can require that described special object makes specific action, thus more contribute to determining the described position of multiple feature key points on described special object.Such as, described specific action can be frontal upright, the back side uprightly, with arms akimbo upright etc.
Described coordinate transformation device 430 can produce the volume coordinate of the multiple specific key point of described special object.Particularly, described volume coordinate is the intrasystem three-dimensional coordinate of physical world coordinates.Described coordinate transformation device 430 based on the imaging ratio of the height of described depth image harvester in physical world and described depth image acquisition component, can produce the volume coordinate of the multiple specific key point of described special object according to described depth image.
According to disclosure embodiment, for each specific key point in depth image, described coordinate transformation device 430 according to the depth value at the horizontal range of the imaging ratio of the described position of specific key point in described depth image, described depth image harvester, described depth image harvester and described special object and described specific key point place, can determine that described specific key point is at the intrasystem three-dimensional coordinate of physical world coordinates.
Alternatively, described coordinate transformation device 430 can set up two-dimensional coordinate system on described depth image, be such as the initial point of described two-dimensional coordinate system with the lower left corner of described depth image, the upper left corner, the lower right corner, the upper right corner or central point, set up two-dimensional coordinate system.On the basis of this two-dimensional coordinate system, increase depth dimension, virtual three-dimensional system of coordinate can be formed.Such as, described coordinate transformation device 430 can according to the imaging ratio of described depth image harvester, the horizontal range of described depth image harvester and described special object, the setting height(from bottom) of described depth image harvester, and in other acquisition parameters of described depth image harvester at least partially, calculate the transformation matrix of coordinates between described virtual three-dimensional system of coordinate and described physical world coordinates system, thus the coordinate position of each specific key point of direct basis in described depth image and depth data can obtain the three-dimensional coordinate of this specific key point in described physical world coordinates system.
Alternatively, described coordinate transformation device 430 can also centered by described depth image harvester, set up image collecting device coordinate system (referred to as camera coordinates system), this camera coordinates is three-dimensional system of coordinate, with the shooting angle of described depth image harvester for depth direction (Z-direction), and construct XY axial plane with the plane vertical with described depth direction.Described coordinate transformation device 430 according to the imaging ratio of described depth image harvester, described depth image harvester and can determine the transformation matrix two-dimensional coordinate of described depth image and corresponding depth value being converted into the three-dimensional coordinate in described camera coordinates system in other acquisition parameters of the horizontal range of described special object, the setting height(from bottom) of described depth image harvester and described depth image harvester at least partially.Then, described coordinate transformation device 430 can according to the direction of the XYZ axle of the direction of the XYZ axle of the setting height(from bottom) of described depth image harvester, described camera coordinates system and described physical world coordinates system, determine the coordinate conversion relation of the three-dimensional coordinate of three-dimensional coordinate in described camera coordinates system and described physical world coordinates system, such as rotational transform relation.
Size calculation device 440, for the volume coordinate of the multiple specific key point based on described special object, calculates the dimension information of described special object.
Such as, when described special object is human body, the dimension information of described special object can be the body information of described special object, and can include but not limited to: height information, brachium information, shoulder breadth information, chest measurement information, waistline information, leg long message, upper body length information, lower-leg length information, thigh circumference information, calf circumference information etc.
Size recommendation apparatus 450, based on the dimension information of described special object, provides size recommendation information.Such as, when described special object is special article, the size recommendation information of the package for holding described special article can be provided according to the size of described special article; When described special object is processing work, size-modify information for improvement of described processing work can be provided according to the size of described special article.Again such as, when described special object is human body, can based on the body information of described human body, the suit length providing described human body is recommended.Described suit length recommendation can be a concrete suit length, or can be an optional suit length scope.
Output unit 460 can export described size recommendation information with video information, audio-frequency information etc.
Described memory storage 470 is for storing described key point geometric model, described transformation matrix of coordinates, described current depth image and described background depth image.In addition, described memory storage 470 is also for storing the computer program code of the method for realizing the object size information that obtains based on depth data according to disclosure embodiment.
In addition, according to disclosure embodiment, additionally provide a kind of computer program, it comprises computer-readable recording medium, and described computer-readable recording medium stores computer program instructions.Described computer program instructions by the method for the object size information of obtaining based on depth data that can realize during computer run according to disclosure embodiment, and/or can realize all or part of function obtaining key point locating device, coordinate transformation device, Size calculation device and size recommendation apparatus in the equipment of object size information based on depth data according to disclosure embodiment.
According to method and apparatus and the computer program of the object size information that obtains based on depth data of disclosure embodiment, by the key point geometric model of pre-defined a certain class object, from depth image, the specific key point of special object is extracted according to described key point geometric model, then the three-dimensional coordinate of described specific key point under physical world coordinates system is determined, thus the critical size information of described special object can be extracted exactly, and obtain the key structural features of described special object.Advantageously, the subsequent operations such as dimension inspection, dimension correction, size recommendation can also be carried out based on extracted critical size information.More advantageously, when described special object is human body, can provide the suit length recommendation of human body according to the body information of human body, this buys clothes for online purchase clothes or solid shop and all has a good application prospect.
The example embodiment of the present disclosure described in detail is above only illustrative, instead of restrictive.It should be appreciated by those skilled in the art that when not departing from principle of the present disclosure and spirit, various amendment can be carried out to these embodiments, combination or sub-portfolio, and such amendment should fall in the scope of the present disclosure.
Claims (20)
1. obtain a method for object size information based on depth data, comprising:
The multiple specific key point of special object is located in depth image;
Produce the volume coordinate of the multiple specific key point of described special object; And
Based on the volume coordinate of the multiple specific key point of described special object, calculate the dimension information of described special object.
2. the multiple specific key point of the method for claim 1, wherein locating special object in depth image comprises:
Described special object is located in depth image; And
Described special object is located described multiple specific key point.
3. method as claimed in claim 2, wherein, described special object is located described multiple specific key point and comprises:
On described special object, locate described multiple specific key point according to key point geometric model, wherein, in described key point geometric model, identify the position of described multiple specific key point.
4. the method for claim 1, wherein described volume coordinate is the intrasystem three-dimensional coordinate of physical world coordinates, and described depth image is gathered by depth image acquisition component,
Wherein, the volume coordinate producing the multiple specific key point of described special object comprises: based on the height of described depth image acquisition component in physical world, shooting angle and imaging ratio, produce the volume coordinate of the multiple specific key point of described special object according to described depth image.
5. the method for claim 1, wherein, described depth image comprises the first depth image and the second depth image, and described first depth image is the direct picture of described special object, and described second depth image is back side image or the side image of described special object.
6. the method for claim 1, wherein, described special object is human body, and described specific key point comprises at least partially following: at the bottom of the crown, neck, left shoulder, left hand elbow, left finesse, right shoulder, right hand elbow, right finesse, chest front center, left stern, left knee, left ankle, left foot, at the bottom of right stern, right knee, right ankle, right crus of diaphragm, left chest edge, right chest edge, left waist edge, right waist edge, waist front center point, lumbar dorsal central point, left hip, right hip, crotch's front center point, buttocks back side central point.
7. method as claimed in claim 6, wherein, described dimension information is the body information of human body, and described body information to comprise in following information at least partially: height information, brachium information, shoulder breadth information, chest measurement information, waistline information, leg long message, upper body length information, lower-leg length information, thigh circumference information, calf circumference information.
8. the method for claim 1, also comprises:
Based on described dimension information, provide size recommendation information.
9. obtain an equipment for object size information based on depth data, comprising:
Depth image harvester, for taking the depth image of special object;
One or more processor;
One or more storer; And
Store computer program instructions in which memory, perform following steps when described computer program instructions is run by described processor:
The multiple specific key point of described special object is located in described depth image;
Produce the volume coordinate of the multiple specific key point of described special object; And
Based on the volume coordinate of the multiple specific key point of described special object, calculate the dimension information of described special object.
10. equipment as claimed in claim 1, wherein, the multiple specific key point of locating special object in depth image comprises:
Described special object is located in depth image; And
Described special object is located described multiple specific key point.
11. equipment as claimed in claim 10, wherein, described special object is located described multiple specific key point and comprises:
On described special object, locate described multiple specific key point according to key point geometric model, wherein, in described key point geometric model, identify the position of described multiple specific key point.
12. equipment as claimed in claim 9, wherein, described volume coordinate is the intrasystem three-dimensional coordinate of physical world coordinates, and described depth image is gathered by depth image acquisition component,
Wherein, the volume coordinate producing the multiple specific key point of described special object comprises: based on the height of described depth image acquisition component in physical world, shooting angle and imaging ratio, produce the volume coordinate of the multiple specific key point of described special object according to described depth image.
13. equipment as claimed in claim 9, wherein, described depth image comprises the first depth image and the second depth image, and described first depth image is the direct picture of described special object, and described second depth image is back side image or the side image of described special object.
14. equipment as claimed in claim 9, wherein, described special object is human body, and described specific key point comprises at least partially following: at the bottom of the crown, neck, left shoulder, left hand elbow, left finesse, right shoulder, right hand elbow, right finesse, chest front center, left stern, left knee, left ankle, left foot, at the bottom of right stern, right knee, right ankle, right crus of diaphragm, left chest edge, right chest edge, left waist edge, right waist edge, waist front center point, lumbar dorsal central point, left hip, right hip, crotch's front center point, buttocks back side central point.
15. equipment as claimed in claim 14, wherein, described dimension information is the body information of human body, and described body information to comprise in following information at least partially: height information, brachium information, shoulder breadth information, chest measurement information, waistline information, leg long message, upper body length information, lower-leg length information, thigh circumference information, calf circumference information.
16. equipment as claimed in claim 9, also perform following steps when described computer program instructions is run by described processor:
Based on described dimension information, provide size recommendation information.
17. equipment as claimed in claim 16, also comprise: output unit, for exporting described size recommendation information.
18. 1 kinds obtain the equipment of object size information based on depth data, comprising:
Depth image harvester, for taking the depth image of special object;
Key point locating device, for locating the multiple specific key point of described special object in described depth image;
Coordinate transformation device, for generation of the volume coordinate of the multiple specific key point of described special object; And
Size calculation device, for the multiple specific key point based on described special object spatial coordinates calculation described in the dimension information of special object.
19. equipment as claimed in claim 18, also comprise: size recommendation apparatus, for providing size recommendation information based on the dimension information of described special object.
20. equipment as claimed in claim 18, also comprise: output unit, for exporting described size recommendation information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290441.4A CN104966284A (en) | 2015-05-29 | 2015-05-29 | Method and equipment for acquiring object dimension information based on depth data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290441.4A CN104966284A (en) | 2015-05-29 | 2015-05-29 | Method and equipment for acquiring object dimension information based on depth data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104966284A true CN104966284A (en) | 2015-10-07 |
Family
ID=54220315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510290441.4A Pending CN104966284A (en) | 2015-05-29 | 2015-05-29 | Method and equipment for acquiring object dimension information based on depth data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104966284A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106327238A (en) * | 2016-08-09 | 2017-01-11 | Tcl移动通信科技（宁波）有限公司 | Method and system for filtering online clothing collocation through user photos |
| CN106529399A (en) * | 2016-09-26 | 2017-03-22 | 深圳奥比中光科技有限公司 | Human body information acquisition method, device and system |
| CN106529400A (en) * | 2016-09-26 | 2017-03-22 | 深圳奥比中光科技有限公司 | Mobile terminal and human body monitoring method and device |
| CN107273846A (en) * | 2017-06-12 | 2017-10-20 | 江西服装学院 | A kind of human somatotype parameter determination method and device |
| CN107481101A (en) * | 2017-07-31 | 2017-12-15 | 广东欧珀移动通信有限公司 | Wear the clothes recommendation method and its device |
| CN107527041A (en) * | 2017-09-08 | 2017-12-29 | 北京奇虎科技有限公司 | Image capture device Real-time Data Processing Method and device, computing device |
| CN108604297A (en) * | 2016-01-19 | 2018-09-28 | 点对点睡眠科技(国际)有限公司 | Device and method for carrying out measurements |
| CN108960985A (en) * | 2018-06-26 | 2018-12-07 | 北京适久科技有限责任公司 | Body parameter generation method and online shopping item recommendation method based on image or video |
| CN109147123A (en) * | 2018-08-03 | 2019-01-04 | 北京旷视科技有限公司 | Unlocking method, device, electronic equipment and the computer storage medium of door-control lock |
| CN110136211A (en) * | 2019-04-18 | 2019-08-16 | 中国地质大学（武汉） | A kind of workpiece localization method and system based on active binocular vision technology |
| CN111179332A (en) * | 2018-11-09 | 2020-05-19 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic device and storage medium |
| CN111426325A (en) * | 2020-06-12 | 2020-07-17 | 北京云迹科技有限公司 | Positioning method and device, robot, storage medium and positioning system |
| CN111435535A (en) * | 2019-01-14 | 2020-07-21 | 株式会社日立制作所 | Method and device for acquiring joint point information |
| CN112884556A (en) * | 2021-03-23 | 2021-06-01 | 中德（珠海）人工智能研究院有限公司 | Shop display method, system, equipment and medium based on mixed reality |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001306900A (en) * | 2000-04-26 | 2001-11-02 | Daishin:Kk | Method for ordering clothes through information communication network |
| CN102682211A (en) * | 2012-05-09 | 2012-09-19 | 晨星软件研发(深圳)有限公司 | Three-dimensional fitting method and device |
| TW201508551A (en) * | 2013-08-30 | 2015-03-01 | Nat Univ Tsing Hua | A non-contact three-dimensional data acquisition system and method body, which on the surface in terms of the system |
-
2015
- 2015-05-29 CN CN201510290441.4A patent/CN104966284A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001306900A (en) * | 2000-04-26 | 2001-11-02 | Daishin:Kk | Method for ordering clothes through information communication network |
| CN102682211A (en) * | 2012-05-09 | 2012-09-19 | 晨星软件研发(深圳)有限公司 | Three-dimensional fitting method and device |
| TW201508551A (en) * | 2013-08-30 | 2015-03-01 | Nat Univ Tsing Hua | A non-contact three-dimensional data acquisition system and method body, which on the surface in terms of the system |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108604297A (en) * | 2016-01-19 | 2018-09-28 | 点对点睡眠科技(国际)有限公司 | Device and method for carrying out measurements |
| CN108604297B (en) * | 2016-01-19 | 2022-02-18 | 点对点睡眠科技(国际)有限公司 | Device and method for carrying out measurements |
| CN106327238A (en) * | 2016-08-09 | 2017-01-11 | Tcl移动通信科技（宁波）有限公司 | Method and system for filtering online clothing collocation through user photos |
| CN106529399A (en) * | 2016-09-26 | 2017-03-22 | 深圳奥比中光科技有限公司 | Human body information acquisition method, device and system |
| CN106529400A (en) * | 2016-09-26 | 2017-03-22 | 深圳奥比中光科技有限公司 | Mobile terminal and human body monitoring method and device |
| CN107273846B (en) * | 2017-06-12 | 2020-08-07 | 江西服装学院 | Human body shape parameter determination method and device |
| CN107273846A (en) * | 2017-06-12 | 2017-10-20 | 江西服装学院 | A kind of human somatotype parameter determination method and device |
| CN107481101A (en) * | 2017-07-31 | 2017-12-15 | 广东欧珀移动通信有限公司 | Wear the clothes recommendation method and its device |
| CN107527041A (en) * | 2017-09-08 | 2017-12-29 | 北京奇虎科技有限公司 | Image capture device Real-time Data Processing Method and device, computing device |
| CN108960985A (en) * | 2018-06-26 | 2018-12-07 | 北京适久科技有限责任公司 | Body parameter generation method and online shopping item recommendation method based on image or video |
| CN109147123B (en) * | 2018-08-03 | 2021-05-04 | 北京旷视科技有限公司 | Unlocking method and device of access control lock, electronic equipment and computer storage medium |
| CN109147123A (en) * | 2018-08-03 | 2019-01-04 | 北京旷视科技有限公司 | Unlocking method, device, electronic equipment and the computer storage medium of door-control lock |
| CN111179332A (en) * | 2018-11-09 | 2020-05-19 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic device and storage medium |
| CN111179332B (en) * | 2018-11-09 | 2023-12-19 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN111435535A (en) * | 2019-01-14 | 2020-07-21 | 株式会社日立制作所 | Method and device for acquiring joint point information |
| CN111435535B (en) * | 2019-01-14 | 2024-03-08 | 株式会社日立制作所 | Method and device for acquiring joint point information |
| CN110136211A (en) * | 2019-04-18 | 2019-08-16 | 中国地质大学（武汉） | A kind of workpiece localization method and system based on active binocular vision technology |
| CN111426325A (en) * | 2020-06-12 | 2020-07-17 | 北京云迹科技有限公司 | Positioning method and device, robot, storage medium and positioning system |
| CN111426325B (en) * | 2020-06-12 | 2020-11-27 | 北京云迹科技有限公司 | Positioning method and device, robot, storage medium and positioning system |
| CN112884556A (en) * | 2021-03-23 | 2021-06-01 | 中德（珠海）人工智能研究院有限公司 | Shop display method, system, equipment and medium based on mixed reality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104966284A (en) | Method and equipment for acquiring object dimension information based on depth data | |
| CN107111833B (en) | Fast 3D model adaptation and anthropometry | |
| US11468612B2 (en) | Controlling display of a model based on captured images and determined information | |
| US20230014804A1 (en) | Processing user selectable product images and facilitating visualization-assisted virtual dressing | |
| CN108475439B (en) | Three-dimensional model generation system, three-dimensional model generation method, and recording medium | |
| EP3479296A1 (en) | System, device, and method of virtual dressing utilizing image processing, machine learning, and computer vision | |
| US10813715B1 (en) | Single image mobile device human body scanning and 3D model creation and analysis | |
| CN109219835A (en) | The generation of the customization wearable article of 3 D-printing | |
| CN107251026A (en) | System and method for generating fictitious situation | |
| CN109427007A (en) | Virtual fit method based on multi-angle of view | |
| CN109214876A (en) | A kind of fitting method and system based on augmented reality | |
| KR102328072B1 (en) | Server for providing virtual fitting service and method using the same | |
| CN109655011B (en) | Method and system for measuring dimension of human body modeling | |
| US11461914B2 (en) | Measuring surface distances on human bodies | |
| CN106773050A (en) | A kind of intelligent AR glasses virtually integrated based on two dimensional image | |
| CN110622218A (en) | Image display method, device, storage medium and terminal | |
| CN111435550A (en) | Image processing method and apparatus, image device, and storage medium | |
| KR20230043346A (en) | System for virtual fashion item-based 3D content service platform | |
| Liu et al. | New anti-blur and illumination-robust combined invariant for stereo vision in human belly reconstruction | |
| CN208497700U (en) | A kind of Image Acquisition modeling | |
| Pei et al. | An image-based measuring technique for the prediction of human body size | |
| WO2021004507A1 (en) | Method and apparatus for controlling virtual character | |
| KR20230043345A (en) | Recording medium for recording the 3D modeling production system | |
| KR20230043342A (en) | Method for creating a 3D-based virtual fitting avatar in the metaverse commerce service | |
| KR20230043349A (en) | Method of providing a virtual character fitting simulation service that reflects the user's characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Yu Gang Inventor after: Li Chao Inventor after: Shang Zeyuan Inventor after: He Qizheng Inventor before: Cao Zhimin Inventor before: Jia Kai |
|
| COR | Change of bibliographic data | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100190 Beijing, Haidian District Academy of Sciences, South Road, No. 2, block A, No. 313 Applicant after: MEGVII INC. Applicant after: Beijing maigewei Technology Co., Ltd. Address before: 100080 room 1001-011, building 3, Haidian street, Beijing, Haidian District, 1 Applicant before: MEGVII INC. Applicant before: Beijing aperture Science and Technology Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151007 |