CN110070542A - Machine learning method, device and the computer readable storage medium of intuition physics - Google Patents
Machine learning method, device and the computer readable storage medium of intuition physics Download PDFInfo
- Publication number
- CN110070542A CN110070542A CN201910365831.1A CN201910365831A CN110070542A CN 110070542 A CN110070542 A CN 110070542A CN 201910365831 A CN201910365831 A CN 201910365831A CN 110070542 A CN110070542 A CN 110070542A
- Authority
- CN
- China
- Prior art keywords
- target object
- network
- intuition
- physics
- machine learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Abstract
The invention discloses a kind of machine learning methods of intuition physics, by obtaining the depth image of target object, the depth image and the first condition vector input generated based on target object characteristic are generated network, obtain the prediction of distortion amount of target object；Then the true strain amount for obtaining target object, in conjunction with true strain amount, determines whether the prediction of distortion amount for generating network output meets expectation using the prediction of distortion amount as the input value for differentiating network；Later when the prediction of distortion amount meets desired, task is executed based on the prediction of distortion amount.The invention also discloses a kind of machine learning device of intuition physics and computer readable storage mediums.The method of the present invention predicts real-time deformation situation of three-dimension object when by external force, using the material property of the object and stress condition as conditional vector, is added to depth variation autocoder and antagonistic training network, realizes real-time Deformation Prediction.
Description
Technical field
The present invention relates to field of artificial intelligence more particularly to a kind of machine learning method of intuition physics, device and
Computer readable storage medium.
Background technique
Key areas of the deep learning as current machine learning suffers from weight in directions such as image recognition, speech analysis
It applies.Deep learning is from neural network evolution, and deep learning is substantially the artificial neural network of deep layer, it is not one
The isolated technology of item, but the synthesis of the multiple fields such as mathematics, statistical machine learning, computer science and artificial neural network.
Up to the present, oneself warp of people is directed to all kinds of different problems such as classification, detection, identification and devises a variety of effective networks
Structure.Since the problems in reality is often more complicated with linear prediction than simply identifying, needed for example, stereo-picture is rebuild
Complete 3D model is exported, intelligent typesetting needs to export complete typesetting scheme, these are that existing neural network cannot be done.
It is still very poor on the generation model of high-dimensional, high complexity.As the landmark of nearly 2 years artificial intelligence fields
It is that higher-dimension generation model is established that generation confrontation network (Wasserstein GAN) is stepped in work, generation confrontation network and moral Leix
Basis, one of them application are exactly the high latitude model of computer vision prediction physical deformation, keep machine portrait people the same, counterweight
The understanding of the real worlds such as power, frictional force, tension becomes possible.
Ambient enviroment is understood and is interacted, is one of the significant capability that robot needs to possess, and including machine
The precondition that many application technologies including people's technology and augmented reality are achieved, especially in prediction target object
In the real-time deformation situation under by external force.It is blocked, interferes however, being frequently run onto target object in practice
The situations such as object and localized loss, difficulty increases when above situation all can cause to predict the deformation of target object in real time,
Such as robot can only be observed when manipulating an object target object part and non-integral.
Meanwhile traditional robot grasp mode regards all target objects as static and hard indeformable rigid body,
Therefore during grab object, there is no the deformations for considering object itself, this is obviously against in physics law, because showing
Most of object in the real world can all generate deformation when by external force.On the other hand, the deformation based on finite element technique point
Although analysis method has degree of precision, it can not be realized by certain depth map and need to spend high operation cost,
It is difficult to complete to predict in real time.It is therefore desirable to propose a kind of machine learning method of intuition physics.
Above content is only used to facilitate the understanding of the technical scheme, and is not represented and is recognized that above content is existing skill
Art.
Summary of the invention
The main purpose of the present invention is to provide a kind of machine learning method of intuition physics, device and computer-readable deposit
Storage media, it is intended to the technical issues of non-rigid object deformation is predicted in solution in real time.
To achieve the above object, the present invention provides a kind of machine learning method of intuition physics, applied to intuition physics
Machine learning system, the machine learning system of the intuition physics include generating network and differentiating network, the intuition physics
Machine learning method the following steps are included:
The depth image for obtaining target object is generated by the depth image of the target object and based on target object characteristic
First condition vector input generate network, obtain the prediction of distortion amount of target object；
The true strain amount for obtaining target object, using the prediction of distortion amount of the generation network output as differentiation network
Input value determines whether the prediction of distortion amount for generating network output meets expectation in conjunction with the true strain amount；
When the prediction of distortion amount of the generation network output meets desired, task is executed based on the prediction of distortion amount.
Preferably, the depth image for obtaining target object by the depth image of the target object and is based on target
The first condition vector of object properties generation inputs the step of generating network, obtaining the prediction of distortion amount of target object
The depth image for obtaining target object, obtains corresponding voxel grid according to the depth image of the target object；
First condition vector is generated based on target object characteristic；
By the voxel grid convert one-dimensional vector in generating network, merge the one-dimensional vector and first condition to
Amount, by generating the prediction of distortion amount of network generation target object and exporting.
Preferably, the depth image for obtaining target object, is corresponded to according to the depth image of the target object
Voxel grid the step of include:
Obtain the 2.5D depth image of target object；
Voxelization processing is carried out to the 2.5D depth image, obtains corresponding voxel grid.
Preferably, described the step of generating first condition vector based on target object characteristic, includes:
Determine the material property of target object and the stress condition of target object；
First condition vector is generated based on the material property and stress condition, the first condition vector is inputted and is generated
Network.
Preferably, the true strain amount for obtaining target object makees the prediction of distortion amount of the generation network output
Determine whether the prediction of distortion amount for generating network output meets the phase in conjunction with the true strain amount for the input value for differentiating network
The step of prestige includes:
Obtain the true strain amount that physical modeler generates；
The prediction of distortion amount input of the generation network output is differentiated into network, in conjunction with the true strain amount and is based on mesh
The second condition vector that object properties generate is marked, determines whether the prediction of distortion amount meets expectation.
Preferably, the depth image for obtaining target object by the depth image of the target object and is based on target
Before the step of first condition vector input that object properties generate generates network, obtains the prediction of distortion amount of target object, institute
State method further include:
Several real-world object deformation patterns are collected, corresponding image data base is established；
It is trained based on described image database to network is generated, so that the predictive ability for generating network gradually increases
By force.
Preferably, described to be trained based on described image database to network is generated, so that described generate the pre- of network
The step of survey ability gradually increases include:
Several voxel grids and conditional vector are obtained based on image data base, if generating by physical modeler by described
Several ground truths pair of dry voxel grid and conditional vector composition；
With several described ground truths pair and differentiate that training generates network to network jointly, continues to optimize the phase for generating network
Parameter is closed, so that the predictive ability for generating network gradually increases.
Preferably, the differentiation network successively includes five warp laminations, wherein each warp lamination includes one and swashs
Function living；
The generation network includes variation autocoder and decoder；
The variation autocoder successively includes five convolutional layers, a full articulamentum and two μ and σ layers,
In, each warp lamination includes an activation primitive；
The decoder successively includes five warp laminations, wherein each warp lamination includes an activation primitive.
In addition, to achieve the above object, the present invention also provides a kind of machine learning device of intuition physics, the intuition object
The machine learning device of reason includes: memory, processor and is stored on the memory and can run on the processor
Intuition physics machine learning program, realized when the machine learning program of the intuition physics is executed by the processor as above
The step of stating the machine learning method of described in any item intuition physics.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium
The machine learning program of intuition physics is stored on storage medium, the machine learning program of the intuition physics is executed by processor
The step of machine learning method of Shi Shixian intuition physics as described in any one of the above embodiments.
The present invention program by the depth image of the target object and is based on by obtaining the depth image of target object
The first condition vector input that target object characteristic generates generates network, obtains the prediction of distortion amount of target object；Then it obtains
The true strain amount of target object, using it is described generation network output prediction of distortion amount as differentiation network input value, in conjunction with
The true strain amount, determines whether the prediction of distortion amount for generating network output meets expectation；It is defeated in the generation network later
When prediction of distortion amount out meets desired, task is executed based on the prediction of distortion amount；The method of the present invention utilizes intuition physics mould
Type predicts real-time deformation situation of three-dimension object when by external force, using the material property of the object and stress condition as item
Part vector is added to depth variation autocoder and antagonistic training network, learns mesh from the material property of target object
The property of standard type enables the network in the method for the present invention to promote on a large scale, mitigates the demand to a large amount of training datas.
Detailed description of the invention
Fig. 1 is belonging to the machine learning device of the intuition physics in the hardware running environment that the embodiment of the present invention is related to
The structural schematic diagram of terminal；
Fig. 2 is the flow diagram of the machine learning method first embodiment of intuition physics of the present invention；
Fig. 3 is the depth image that target object is obtained in the machine learning method second embodiment of intuition physics of the present invention,
The depth image of the target object and the first condition vector input generated based on target object characteristic are generated into network, obtained
The refinement flow diagram of the step of prediction of distortion amount of target object
Fig. 4 is the structural schematic diagram in the machine learning method second embodiment of intuition physics of the present invention；
Fig. 5 is the true strain that target object is obtained in the machine learning method 3rd embodiment of intuition physics of the present invention
Amount is determined using the prediction of distortion amount of the generation network output as the input value for differentiating network in conjunction with the true strain amount
Whether the prediction of distortion amount for generating network output meets the refinement flow diagram of the step of desired；
Fig. 6 is the flow diagram in the machine learning method fourth embodiment of intuition physics of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
As shown in Figure 1, the affiliated terminal structure of device that Fig. 1 is the hardware running environment that the embodiment of the present invention is related to shows
It is intended to.
The terminal of that embodiment of the invention can be PC, be also possible to smart phone, tablet computer, E-book reader, MP3
(Moving Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio level 3)
Player, MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard sound
Frequency level 3) the packaged type terminal device having a display function such as player, portable computer.
As shown in Figure 1, the terminal may include: processor 1001, such as CPU, network interface 1004, user interface
1003, memory 1005, communication bus 1002.Wherein, communication bus 1002 is for realizing the connection communication between these components.
User interface 1003 may include display screen (Display), input unit such as keyboard (Keyboard), optional user interface
1003 can also include standard wireline interface and wireless interface.Network interface 1004 optionally may include that the wired of standard connects
Mouth, wireless interface (such as WI-FI interface).Memory 1005 can be high speed RAM memory, be also possible to stable memory
(non-volatile memory), such as magnetic disk storage.Memory 1005 optionally can also be independently of aforementioned processor
1001 storage device.
Optionally, terminal can also include camera, RF (Radio Frequency, radio frequency) circuit, sensor, audio
Circuit, WiFi module etc..Wherein, sensor such as optical sensor, motion sensor and other sensors.Specifically, light
Sensor may include ambient light sensor and proximity sensor, wherein ambient light sensor can according to the light and shade of ambient light come
The brightness of display screen is adjusted, proximity sensor can close display screen and/or backlight when mobile terminal is moved in one's ear.As
One kind of motion sensor, gravity accelerometer can detect the size of (generally three axis) acceleration in all directions, quiet
Size and the direction that can detect that gravity when only, the application that can be used to identify mobile terminal posture are (such as horizontal/vertical screen switching, related
Game, magnetometer pose calibrating), Vibration identification correlation function (such as pedometer, tap) etc.；Certainly, mobile terminal can also match
The other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor are set, details are not described herein.
It will be understood by those skilled in the art that the restriction of the not structure paired terminal of terminal structure shown in Fig. 1, can wrap
It includes than illustrating more or fewer components, perhaps combines certain components or different component layouts.
As shown in Figure 1, as may include that operating system, network are logical in a kind of memory 1005 of computer storage medium
Believe module, the machine learning program of Subscriber Interface Module SIM and intuition physics.
In terminal shown in Fig. 1, network interface 1004 is mainly used for connecting background server, carries out with background server
Data communication；User interface 1003 is mainly used for connecting client (user terminal), carries out data communication with client；And processor
1001 can be used for calling the machine learning program of the intuition physics stored in memory 1005.
In the present embodiment, the machine learning device of intuition physics includes: memory 1005, processor 1001 and is stored in
On the memory 1005 and the machine learning program of intuition physics that can be run on the processor 1001, wherein processing
When device 1001 calls the machine learning program of the intuition physics stored in memory 1005, and execute following operation:
The depth image for obtaining target object is generated by the depth image of the target object and based on target object characteristic
First condition vector input generate network, obtain the prediction of distortion amount of target object；
The true strain amount for obtaining target object, using the prediction of distortion amount of the generation network output as differentiation network
Input value determines whether the prediction of distortion amount for generating network output meets expectation in conjunction with the true strain amount；
When the prediction of distortion amount of the generation network output meets desired, task is executed based on the prediction of distortion amount.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
The depth image for obtaining target object, obtains corresponding voxel grid according to the depth image of the target object；
First condition vector is generated based on target object characteristic；
By the voxel grid convert one-dimensional vector in generating network, merge the one-dimensional vector and first condition to
Amount, by generating the prediction of distortion amount of network generation target object and exporting.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
Obtain the 2.5D depth image of target object；
Voxelization processing is carried out to the 2.5D depth image, obtains corresponding voxel grid.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
Determine the material property of target object and the stress condition of target object；
First condition vector is generated based on the material property and stress condition, the first condition vector is inputted and is generated
Network.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
Obtain the true strain amount that physical modeler generates；
The prediction of distortion amount input of the generation network output is differentiated into network, in conjunction with the true strain amount and is based on mesh
The second condition vector that object properties generate is marked, determines whether the prediction of distortion amount meets expectation.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
Several real-world object deformation patterns are collected, corresponding image data base is established；
It is trained based on described image database to network is generated, so that the predictive ability for generating network gradually increases
By force.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
Several voxel grids and conditional vector are obtained based on image data base, if generating by physical modeler by described
Several ground truths pair of dry voxel grid and conditional vector composition；
With several described ground truths pair and differentiate that training generates network to network jointly, continues to optimize the phase for generating network
Parameter is closed, so that the predictive ability for generating network gradually increases.
Further, processor 1001 can call the machine learning program of the intuition physics stored in memory 1005,
Also execute following operation:
The generation network includes variation autocoder and decoder；
The variation autocoder successively includes five convolutional layers, a full articulamentum and two μ and σ layers,
In, each warp lamination includes an activation primitive；
The decoder successively includes five warp laminations, wherein each warp lamination includes an activation primitive.
First embodiment of the invention provides a kind of machine learning method of intuition physics, is that the present invention is straight referring to Fig. 2, Fig. 2
Feel that the flow diagram of the machine learning method first embodiment of physics, the machine learning method of the intuition physics include:
Step S10 obtains the depth image of target object, by the depth image of the target object and is based on target object
The first condition vector input that characteristic generates generates network, obtains the prediction of distortion amount of target object；
Depth image is also range image, refer to using distance (depth) value of each point in from image acquisition device to scene as
The image of pixel value.Acquisition methods have: laser radar Depth Imaging method, computer stereo vision imaging, coordinate measuring machine method, not
That the Schlieren method, Structure light method.The material property of the target object includes Young's modulus and Poisson's ratio, the target object by
Power situation includes amount of force and position of action point.Conditional vector is to carry out discretization to multiple physical conditions, is obtained multiple
Corresponding numerical value set, is indicated with the form of vector.For example, when robot needs to judge the deflection of target obstacle,
Obtain the size of the power applied to it, it, can be by the way that these three features be compiled when the material of the active position of power and barrier
Code is only hot vector (f, a, m), including each feature including f, a, m indicates the condition of discretization with binary form；
Fixed number of bits each feature can also be arranged.
It in the present embodiment, is the depth image that target object is got by the shooting function of robot itself, this
Image not only includes target object, further includes surrounding environment.Depth image passes through certain processing, such as coordinate is converted,
The point cloud data that may be calculated point cloud data, while meeting certain condition can also be using inverse as depth image.Obtain object
The depth image of body obtains corresponding first voxel grid according to the depth image of the target object；Based on target object spy
Property generate first condition vector；One-dimensional vector is converted by first voxel grid in generating network, is merged described one-dimensional
Vector and first condition vector, by generating the prediction of distortion amount of network generation target object and exporting.
The basic premise that modeling is autonomous robot is carried out to the physical characteristic of everyday objects.It is new we have proposed one
Confrontation network is generated, it can be from deformation of the single RGB-D image prediction target object under by external force.The network base
Network is fought in generating, and is trained on the different object sets generated by physics finite element model simulator.We
Method inherits the extensive attribute for generating confrontation network.This means that network can be the case where giving the single depth views of object
The entire three-dimensional appearance of lower reconstructed object.It generates network to be mainly made of autocoder and decoder, for the ease of rebuilding body
The internal structure of plain grid, the present invention in autocoder between traditional encoder and decoder have great-jump-forward connect
It connects.
Step S20 obtains the true strain amount of target object, using the prediction of distortion amount of the generation network output as sentencing
The input value of other network determines whether the prediction of distortion amount for generating network output meets expectation in conjunction with the true strain amount；
Physical modeler (physics simulator) is also physical engine (physics engine), can simulate true
The rule of various object of which movement in the real world.Preset picture database can be stored in the memory of itself by physical engine
In, when being trained to generation network, several opposite true value pair are generated based on picture database, the ground truth is to by three
Dimension point cloud and conditional vector composition, convert corresponding voxel grid by software for three-dimensional point cloud later.Wherein, three-dimensional point cloud
The magnanimity point set for referring to target surface characteristic, the point cloud obtained according to laser measurement principle, including three-dimensional coordinate and laser are anti-
Intensity is penetrated, the point cloud obtained according to photogrammetry principles, including three-dimensional coordinate and colouring information, it is surveyed in conjunction with laser measurement and photography
Amount principle obtains a cloud, including three-dimensional coordinate, laser reflection intensity and colouring information, is obtaining each down-sampling point of body surface
Space coordinate after, what is obtained is the set of a point, referred to as " point cloud ".
Obtain the true strain amount that physical modeler generates；The prediction of distortion amount of the generation network output is inputted and is differentiated
Network determines the prediction of distortion in conjunction with the true strain amount and the second condition vector generated based on target object characteristic
Whether amount meets expectation.Second condition vector is modified so as to the prediction of distortion amount with voxel grid form and really
Deflection is connected.Since the deflection of prediction target object is a high-dimensional problem, differentiate that network can export one
Intensive vector differentiates the similarity as a result, namely between prediction of distortion amount and true strain amount to represent.
Step S30 is held when the prediction of distortion amount of the generation network output meets desired based on the prediction of distortion amount
Row task.
When getting the prediction of distortion amount with generation network output, can executing for task includes: manipulation task, crawl
Task, ground deformation assessment and other in the end-to-end academic environment of intuitive physical model predictive power to non-rigid object
Influence task.
It interacts what robot and circumstances not known generated without constraint, can possess as the common-sense mankind's intuition pushes away
Reason ability be it is necessary, can especially know target object generated reaction when by external force.Have with traditional
First method is limited on the contrary, our method is sufficiently fast, can be used for applying in real time.
The machine learning method of the intuition physics proposed in the present embodiment will by obtaining the depth image of target object
The depth image of the target object and the first condition vector input generated based on target object characteristic generate network, obtain mesh
Mark the prediction of distortion amount of object；Then the true strain amount for obtaining target object, by the prediction of distortion of the generation network output
It measures as the input value for differentiating network, in conjunction with the true strain amount, determines whether the prediction of distortion amount for generating network output accords with
Close expectation；Later when the prediction of distortion amount of the generation network output meets desired, is executed and appointed based on the prediction of distortion amount
Business；The method of the present invention predicts real-time deformation situation of three-dimension object when by external force using intuition physical model, by the object
The material property and stress condition of body are added to depth variation autocoder and antagonistic training network as conditional vector,
The property for learning objective body from the material property of target object, enables the network in the method for the present invention to push away on a large scale
Extensively, mitigate the demand to a large amount of training datas.
Based on first embodiment, the second embodiment of the machine learning method of intuition physics of the present invention is proposed, reference Fig. 3,
Step S10 includes:
Step S11 obtains the depth image of target object, obtains corresponding according to the depth image of the target object
One voxel grid；
Step S12 generates first condition vector based on target object characteristic；
Step S13 converts one-dimensional vector for first voxel grid in generating network, merges the one-dimensional vector
With first condition vector, the prediction of distortion amount of target object is generated by generation network and is exported.
Obtain the 2.5D depth image of target object；Voxelization processing is carried out to the 2.5D depth image, is corresponded to
The first voxel grid.Voxelization processing is the process for converting depth image to voxel grid, voxel grid namely one
3D point cloud.The definition of point cloud is from when beam of laser is irradiated to body surface, and the laser reflected can carry orientation, distance
Etc. information.If laser beam is scanned according to certain track, the laser point information of reflection will be recorded in scanning, due to
Scanning is extremely fine, then can obtain a large amount of laser point, thus can form laser point cloud.Point cloud format has * .las；*
.pcd；* .txt etc..Depth image may be calculated point cloud data by coordinate conversion；Regular and necessary information point cloud number
According to can be using inverse as depth image.
Determine the material property of target object and the stress condition of target object；Based on the material property and stress
Situation generates first condition vector, and the first condition vector is inputted and generates network.The material property of target object includes mesh
The Young's modulus and Poisson's ratio of object materials are marked, the stress condition of target object includes the size and work of external force suffered by target object
With point.
As shown in figure 4, the generation network includes variation autocoder and decoder；The variation autocoder according to
Secondary includes five convolutional layers, a full articulamentum and two μ and σ layers, wherein each warp lamination includes an activation
Function；The decoder successively includes five warp laminations, wherein each warp lamination includes an activation primitive.
First voxel grid is inputted after generating network, initially enters and divide autocoder part partially, available one
One-dimensional vector is merged, one after being merged one with based on first condition vector made of target object characteristic encoding
Dimensional vector, input decoder obtain the prediction of distortion amount generated by decoder processes.
The machine learning method of the intuition physics proposed in the present embodiment, by obtaining the depth image of target object, root
Corresponding voxel grid is obtained according to the depth image of the target object；Be then based on target object characteristic generate first condition to
Amount；By the voxel grid convert one-dimensional vector in generating network later, merge the one-dimensional vector and first condition to
Amount, by generating the prediction of distortion amount of network generation target object and exporting；The present invention is deep by the 2.5D for obtaining target object
Degree figure predicts its deformation generated in turn, obtains training data based on physical modeler, makes a living into network and provide prediction data
And training data.
Based on first embodiment, the 3rd embodiment of the machine learning method of intuition physics of the present invention is proposed, reference Fig. 5,
Step S20 includes:
Step S21 obtains the true strain amount that physical modeler generates；
The prediction of distortion amount input of the generation network output is differentiated network, in conjunction with the true strain amount by step S22
With the second condition vector generated based on target object characteristic, determine whether the prediction of distortion amount meets expectation.
As shown in figure 4, differentiating that network successively includes five warp laminations, wherein each warp lamination includes one and swashs
Function living.
Physical modeler can simulate the rule of various object of which movement in real world.Physical engine can be by preset figure
Sheet data library is stored in the memory of itself, when being trained to generation network, generates several based on picture database
Opposite true value pair, the ground truth are formed to by three-dimensional point cloud and conditional vector, later convert three-dimensional point cloud to by software
Corresponding voxel grid.Obtain the true strain amount that physical modeler generates；By the prediction of distortion amount of the generation network output
Input differentiates network,
In conjunction with the true strain amount and the second condition vector generated based on target object characteristic, determine that the prediction becomes
Whether shape amount meets expectation.Second condition vector be modified so as to prediction of distortion amount with voxel grid form and true
Real deflection is connected.Since the deflection of prediction target object is a high-dimensional problem, differentiate that network can export one
A intensive vector differentiates the similarity as a result, namely between prediction of distortion amount and true strain amount to represent.
The machine learning method of the intuition physics proposed in the present embodiment, the true change generated by obtaining physical modeler
Shape amount；Then the prediction of distortion amount input of the generation network output is differentiated into network, in conjunction with the true strain amount and be based on
The second condition vector that target object characteristic generates, determines whether the prediction of distortion amount meets expectation；This method realizes logical
Differentiation network integration second condition vector sum true strain amount is crossed, determines whether the prediction of distortion amount for generating network output meets the phase
The technical effect of prestige.
Based on first embodiment, the fourth embodiment of the machine learning method of intuition physics of the present invention is proposed, reference Fig. 6,
Before step S10, the method also includes:
Step S40 collects several real-world object deformation patterns, establishes corresponding image data base；
Step S50 is trained based on described image database to network is generated, so that the prediction energy for generating network
Power gradually increases.
Several voxel grids and conditional vector are obtained based on image data base, if generating by physical modeler by described
Several ground truths pair of dry voxel grid and conditional vector composition；With several described ground truths pair and differentiate network
Common training generates network, continues to optimize the relevant parameter for generating network, so that the predictive ability for generating network gradually increases
By force.
Physical modeler can simulate the rule of various object of which movement in real world, can be by preset picture database
It is stored in the memory of itself, when being trained to generation network, several opposite true value is generated based on picture database
Right, which forms to by three-dimensional point cloud and conditional vector, converts corresponding body by software for three-dimensional point cloud later
Plain grid.By the ground truth of physical engine generation to generation network is input to, in the predictive information for obtaining generating network output
When, by the predictive information and above-mentioned ground truth to as the input for differentiating network, predictive information be exactly by generation network over the ground
Face true value is to the predicted value estimated according to certain rule.Later by between differentiation network query function predictive information and ground truth pair
Difference, updated according to the difference differentiate network loss function in corresponding parameter.Meanwhile more according to above-mentioned loss function
The relevant parameter of newly-generated network, so that the predictive ability for generating network gradually increases, that is, the predictive information exported is more next
Closer to the ground truth pair of input, reach the degree for differentiating that network can not be differentiated.
The machine learning method of the intuition physics proposed in the present embodiment, by collecting several real-world object deformation patterns,
Establish corresponding image data base；It is then based on described image database and is trained to network is generated, so that the generation net
The predictive ability of network gradually increases；The present invention uses the structure system of cascade form, and conditional vector can be made as generation network
A part with network inputs value is differentiated, realizes effective network training mode.
In addition, the embodiment of the present invention also proposes a kind of computer readable storage medium, the computer readable storage medium
On be stored with the machine learning program of intuition physics, realized such as when the machine learning program of the intuition physics is executed by processor
Lower operation:
The depth image for obtaining target object is generated by the depth image of the target object and based on target object characteristic
First condition vector input generate network, obtain the prediction of distortion amount of target object；
The true strain amount for obtaining target object, using the prediction of distortion amount of the generation network output as differentiation network
Input value determines whether the prediction of distortion amount for generating network output meets expectation in conjunction with the true strain amount；
When the prediction of distortion amount of the generation network output meets desired, task is executed based on the prediction of distortion amount.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
The depth image for obtaining target object, obtains corresponding voxel grid according to the depth image of the target object；
First condition vector is generated based on target object characteristic；
By the voxel grid convert one-dimensional vector in generating network, merge the one-dimensional vector and first condition to
Amount, by generating the prediction of distortion amount of network generation target object and exporting.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
Obtain the 2.5D depth image of target object；
Voxelization processing is carried out to the 2.5D depth image, obtains corresponding voxel grid.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
Determine the material property of target object and the stress condition of target object；
First condition vector is generated based on the material property and stress condition, the first condition vector is inputted and is generated
Network.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
Obtain the true strain amount that physical modeler generates；
The prediction of distortion amount input of the generation network output is differentiated into network, in conjunction with the true strain amount and is based on mesh
The second condition vector that object properties generate is marked, determines whether the prediction of distortion amount meets expectation.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
Several real-world object deformation patterns are collected, corresponding image data base is established；
It is trained based on described image database to network is generated, so that the predictive ability for generating network gradually increases
By force.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
Several voxel grids and conditional vector are obtained based on image data base, if generating by physical modeler by described
Several ground truths pair of dry voxel grid and conditional vector composition；
With several described ground truths pair and differentiate that training generates network to network jointly, continues to optimize the phase for generating network
Parameter is closed, so that the predictive ability for generating network gradually increases.
Further, following operation is also realized when the machine learning program of the intuition physics is executed by processor:
The differentiation network successively includes five warp laminations, wherein each warp lamination includes an activation primitive；
The generation network includes variation autocoder and decoder；
The variation autocoder successively includes five convolutional layers, a full articulamentum and two μ and σ layers,
In, each warp lamination includes an activation primitive；
The decoder successively includes five warp laminations, wherein each warp lamination includes an activation primitive.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or the system that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or system institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do
There is also other identical elements in the process, method of element, article or system.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of machine learning method of intuition physics, the machine learning system applied to intuition physics, which is characterized in that described
The machine learning system of intuition physics include generate network and differentiate network, the machine learning method of the intuition physics include with
Lower step:
The depth image for obtaining target object, by the depth image of the target object and generated based on target object characteristic the
The input of one conditional vector generates network, obtains the prediction of distortion amount of target object；
The true strain amount for obtaining target object, using the prediction of distortion amount of the generation network output as the input for differentiating network
Value determines whether the prediction of distortion amount for generating network output meets expectation in conjunction with the true strain amount；
When the prediction of distortion amount of the generation network output meets desired, task is executed based on the prediction of distortion amount.
2. the machine learning method of intuition physics as described in claim 1, which is characterized in that the depth for obtaining target object
Image is spent, the depth image of the target object and the first condition vector input generated based on target object characteristic are generated into net
Network, the step of obtaining the prediction of distortion amount of target object include:
The depth image for obtaining target object, obtains corresponding voxel grid according to the depth image of the target object；
First condition vector is generated based on target object characteristic；
One-dimensional vector is converted by the voxel grid in generating network, merges the one-dimensional vector and first condition vector,
By generating the prediction of distortion amount of network generation target object and exporting.
3. the machine learning method of intuition physics as claimed in claim 2, which is characterized in that the depth for obtaining target object
Image is spent, the step of obtaining corresponding voxel grid according to the depth image of the target object includes:
Obtain the 2.5D depth image of target object；
Voxelization processing is carried out to the 2.5D depth image, obtains corresponding voxel grid.
4. the machine learning method of intuition physics as claimed in claim 2, which is characterized in that described to be based on target object characteristic
Generate first condition vector the step of include:
Determine the material property of target object and the stress condition of target object；
First condition vector is generated based on the material property and stress condition.
5. the machine learning method of intuition physics as described in claim 1, which is characterized in that described to obtain the true of target object
Real deflection, using the prediction of distortion amount of the generation network output as the input value for differentiating network, in conjunction with the true strain
Amount, determines whether the prediction of distortion amount for generating network output meets the step of desired and include:
Obtain the true strain amount that physical modeler generates；
The prediction of distortion amount input of the generation network output is differentiated into network, in conjunction with the true strain amount and is based on object
The second condition vector that bulk properties generates, determines whether the prediction of distortion amount meets expectation.
6. the machine learning method of intuition physics as described in claim 1, which is characterized in that the depth for obtaining target object
Image is spent, the depth image of the target object and the first condition vector input generated based on target object characteristic are generated into net
Network, before the step of obtaining the prediction of distortion amount of target object, the method also includes:
Several real-world object deformation patterns are collected, corresponding image data base is established；
It is trained based on described image database to network is generated, so that the predictive ability for generating network gradually increases.
7. the machine learning method of intuition physics as claimed in claim 6, which is characterized in that described to be based on described image data
Library is trained to network is generated, so that the predictive ability for generating network includes: the step of gradually increasing
Obtain several voxel grids and conditional vector based on image data base, by physical modeler generate by it is described several
Several ground truths pair of voxel grid and conditional vector composition；
With several described ground truths pair and differentiate that training generates network to network jointly, continues to optimize the related ginseng for generating network
Number, so that the predictive ability for generating network gradually increases.
8. the machine learning method of intuition physics as described in claim 1, which is characterized in that the differentiation network successively includes
Five warp laminations, wherein each warp lamination includes an activation primitive；
The generation network includes variation autocoder and decoder；
The variation autocoder successively includes five convolutional layers, a full articulamentum and two μ and σ layers, wherein every
A warp lamination all includes an activation primitive；
The decoder successively includes five warp laminations, wherein each warp lamination includes an activation primitive.
9. a kind of machine learning device of intuition physics, which is characterized in that the machine learning device of the intuition physics includes: to deposit
Reservoir, processor and the machine learning journey for being stored in the intuition physics that can be run on the memory and on the processor
It is realized as described in any one of claims 1 to 8 when the machine learning program of sequence, the intuition physics is executed by the processor
Intuition physics machine learning method the step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with intuition object on the computer readable storage medium
It is realized when the machine learning program of the machine learning program of reason, the intuition physics is executed by processor as in claim 1 to 8
The step of machine learning method of described in any item intuition physics.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910365831.1A CN110070542A (en) | 2019-04-30 | 2019-04-30 | Machine learning method, device and the computer readable storage medium of intuition physics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910365831.1A CN110070542A (en) | 2019-04-30 | 2019-04-30 | Machine learning method, device and the computer readable storage medium of intuition physics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110070542A true CN110070542A (en) | 2019-07-30 |
Family
ID=67369814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910365831.1A Pending CN110070542A (en) | 2019-04-30 | 2019-04-30 | Machine learning method, device and the computer readable storage medium of intuition physics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110070542A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017120897A1 (en) * | 2016-01-15 | 2017-07-20 | 武汉武大卓越科技有限责任公司 | Object surface deformation feature extraction method based on line scanning three-dimensional point cloud |
| CN108021131A (en) * | 2017-11-28 | 2018-05-11 | 王智华 | A kind of robot vision recognition methods, device and computer-readable recording medium |
| CN108510573A (en) * | 2018-04-03 | 2018-09-07 | 南京大学 | A method of the multiple views human face three-dimensional model based on deep learning is rebuild |
| CN109102078A (en) * | 2018-07-11 | 2018-12-28 | 北京墨丘科技有限公司 | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment |
-
2019
- 2019-04-30 CN CN201910365831.1A patent/CN110070542A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017120897A1 (en) * | 2016-01-15 | 2017-07-20 | 武汉武大卓越科技有限责任公司 | Object surface deformation feature extraction method based on line scanning three-dimensional point cloud |
| CN108021131A (en) * | 2017-11-28 | 2018-05-11 | 王智华 | A kind of robot vision recognition methods, device and computer-readable recording medium |
| CN108510573A (en) * | 2018-04-03 | 2018-09-07 | 南京大学 | A method of the multiple views human face three-dimensional model based on deep learning is rebuild |
| CN109102078A (en) * | 2018-07-11 | 2018-12-28 | 北京墨丘科技有限公司 | A kind of optimization method of machine learning, system, computer storage medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| ZHIHUA WANG 等: "3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations", 《ARXIV:1805.00328V2》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108021131B (en) | Robot vision identification method and device and computer readable storage medium | |
| Movshovitz-Attias et al. | How useful is photo-realistic rendering for visual learning? | |
| CN111626218B (en) | Image generation method, device, equipment and storage medium based on artificial intelligence | |
| Gao et al. | Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations | |
| CN105378796B (en) | Scalable volume 3D reconstruct | |
| CN100407798C (en) | Three-dimensional geometric mode building system and method | |
| CN105051755A (en) | Part and state detection for gesture recognition | |
| CN113362382A (en) | Three-dimensional reconstruction method and three-dimensional reconstruction device | |
| JP2016218999A (en) | Method for training classifier to detect object represented in image of target environment | |
| CN110399809A (en) | The face critical point detection method and device of multiple features fusion | |
| CN112085835B (en) | Three-dimensional cartoon face generation method and device, electronic equipment and storage medium | |
| Adate et al. | A survey on deep learning methodologies of recent applications | |
| CN112132739A (en) | 3D reconstruction and human face posture normalization method, device, storage medium and equipment | |
| CN108460152A (en) | A kind of the space-filling curve method, apparatus and computer readable storage medium of data | |
| CN115131849A (en) | Image generation method and related device | |
| JP2021131854A (en) | Basic learning of deformation | |
| CN115222896A (en) | Three-dimensional reconstruction method and device, electronic equipment and computer-readable storage medium | |
| US20220229943A1 (en) | Joint retrieval and mesh deformation | |
| Bakhat et al. | Katz centrality based approach to perform human action recognition by using OMKZ | |
| CN116012626B (en) | Material matching method, device, equipment and storage medium for building elevation image | |
| CN115953330B (en) | Texture optimization method, device, equipment and storage medium for virtual scene image | |
| CN110084845A (en) | Deformation Prediction method, apparatus and computer readable storage medium | |
| Gazani et al. | Bag of views: An appearance-based approach to next-best-view planning for 3d reconstruction | |
| CN110070542A (en) | Machine learning method, device and the computer readable storage medium of intuition physics | |
| CN112926681B (en) | Target detection method and device based on deep convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190730 |