CN110216671A - A kind of mechanical gripper training method and system based on Computer Simulation - Google Patents
A kind of mechanical gripper training method and system based on Computer Simulation Download PDFInfo
- Publication number
- CN110216671A CN110216671A CN201910398732.3A CN201910398732A CN110216671A CN 110216671 A CN110216671 A CN 110216671A CN 201910398732 A CN201910398732 A CN 201910398732A CN 110216671 A CN110216671 A CN 110216671A
- Authority
- CN
- China
- Prior art keywords
- crawled
- environment
- handgrip
- model
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000005094 computer simulation Methods 0.000 title claims abstract description 20
- 238000004088 simulation Methods 0.000 claims abstract description 67
- 239000000126 substance Substances 0.000 claims abstract description 48
- 238000010801 machine learning Methods 0.000 claims abstract description 11
- 238000013473 artificial intelligence Methods 0.000 claims description 42
- 238000010276 construction Methods 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 6
- 230000000704 physical effect Effects 0.000 claims description 5
- 239000000463 material Substances 0.000 description 22
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000005484 gravity Effects 0.000 description 5
- 238000009825 accumulation Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003746 surface roughness Effects 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 3
- 230000006378 damage Effects 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/1605—Simulation of manipulator lay-out, design, modelling of manipulator
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
Abstract
The present embodiments relate to a kind of mechanical gripper training method and system based on Computer Simulation, wherein the described method includes: respectively building emulation be crawled substance environment and handgrip environment；Using the two-dimensional image data being crawled in substance environment described in the sensor Simulation acquisition in the handgrip environment, and determine that at least one is crawled the three-dimensional information of object according to the two-dimensional image data；According to the three-dimensional information for being crawled object, the handgrip in the handgrip environment is controlled to the machine learning training for being crawled object and being emulated crawl at least once, to obtain the Optimal Grasp model parameter being currently crawled under substance environment.Technical solution provided by the present application can be improved the efficiency and adaptability of manipulator crawl training.
Description
Technical field
This application involves technical field of data processing, in particular to a kind of mechanical gripper training side based on Computer Simulation
Method and system.
Background technique
Existing mechanical gripper is developed generally be directed to a fixed task.This means that the crawl control of manipulator
Method processed is all pre-designed and debugs.However, with the automation demanding of manufacturing industry large area, rapid deployment mechanical arm
Just become new research direction.In general, needing to carry out mechanical arm for specific task, such as object to be captured
The debugging of control method.The prior art needs to debug mechanical gripper for a task, such as a specific factory, so that
Grasping means can adapt to the task.This process generally requires the longer time period, and it is limited to grab number.
It makes repeated attempts in actual scene often by material object in the prior art to carry out the debugging of manipulator, with
Determine the ideal grasp mode that the material object is directed under the scene.Although this mode can solve certain problem, but
It is affected, and debugging result is more by being limited, is only capable of for specific scene to specific object by artificial experience first
Body realizes effectively crawl.Meanwhile existing debud mode operates material object repeatedly, will necessarily repeatedly fail before completion, pole
Handgrip easy to damage or crawl object；And every time for restore debugging scenarios need the work of manpower intervention more, realize efficiency and at
This is undesirable.
Computer simulation technique can carry out certain convenience to manipulator training band, but existing emulation mode is excessively relied on and imitated
The authenticity of true environmental simulation, allows training result to be difficult to adapt to true environment instead.Typically, if directly using imitative when training
Genuine 3 dimension module data actually fail to make the object identification ability of manipulator to be trained, and practical application effect is poor；Its
It is secondary, analogue simulation can not all real scene of exhaustion, for failing the new object new scene trained up, existing way is suitable
Should be able to power difference fault rate it is high, easily damage is crawled object.
Summary of the invention
The application's is designed to provide a kind of mechanical gripper training method and system based on Computer Simulation, Neng Gouti
The efficiency and adaptability of high manipulator crawl training.
To achieve the above object, the application provides a kind of mechanical gripper training method based on Computer Simulation, the side
Method includes:
What building emulated respectively is crawled substance environment and handgrip environment；
Using the two-dimensional image data being crawled described in the sensor Simulation acquisition in the handgrip environment in substance environment, and
Determine that at least one is crawled the three-dimensional information of object according to the two-dimensional image data；
According to the three-dimensional information for being crawled object, the handgrip controlled in the handgrip environment carries out the object that is crawled
The machine learning training of crawl is emulated, at least once to obtain the Optimal Grasp model parameter being currently crawled under substance environment.
Further, it is described respectively building emulation be crawled substance environment and handgrip environment includes:
Establish each three-dimensional simulation model for being crawled object and each handgrip respectively according to physical samples, wherein described three
Tie up multiple actual physical properties that simulation model has the corresponding physical samples；
It chooses multiple three-dimensional simulation models for being crawled object and establishes at random and be crawled substance environment；
Choose at least one handgrip three-dimensional simulation model it is corresponding described in be crawled substance environment setting and establish the handgrip ring
Border；Wherein, the data for being crawled substance environment and the handgrip environment are mutually indepedent.
Further, the method also includes:
Each three-dimensional simulation model for being crawled object is established according to physical samples, and is simulated by the three-dimensional simulation model
The surface image that sensor obtains is established between the surface image and the three-dimensional simulation model and/or the physical samples
Incidence relation；
It is closed using the association of the three-dimensional simulation model, the two-dimensional image data that the sensor acquires and foundation
System is trained neural network model, obtains the first artificial intelligence mould for identifying three-dimension object according to two-dimensional image data
Type；Wherein, first artificial intelligence model can identify two-dimensional image data to be identified, described to be identified to obtain
The corresponding three-dimensional information for being crawled object of two-dimensional image data.
Further, the crawl model includes the second artificial intelligence model of planning crawl strategy and carries out handgrip control
Third artificial intelligence model；Wherein,
Second artificial intelligence model is planned according to the two-dimensional image data and the three-dimensional information for being crawled object
Export crawl task and implementation strategy；
The third artificial intelligence model controls the output of handgrip driving unit according to the implementation strategy, makes the handgrip
It grabs and is crawled object to realize the crawl task described in moving.
Further, when the method is applied to the Training scene of multiple handgrips, second artificial intelligence model
Output further includes handgrip selection strategy.
Further, the method also includes:
It such as can only obtain the fetching that index is lower than preset threshold in true environment, then by true picture and according to institute
The three-dimensional information for stating the object of true picture acquisition inputs online and the environment of corresponding building emulation, carries out to the crawl model
It trains in real time, and determines the fetching of the true environment by result trained in real time.
To achieve the above object, the application also provides a kind of mechanical gripper training system based on Computer Simulation, described
System includes:
Environment construction unit is crawled substance environment and handgrip environment for construct emulation respectively；
Simulation unit, for utilizing two be crawled in substance environment described in the sensor Simulation acquisition in the handgrip environment
Dimensional data image, and determine that at least one is crawled the three-dimensional information of object according to the two-dimensional image data；
Training unit controls the handgrip in the handgrip environment to institute for being crawled the three-dimensional information of object according to
It states and is crawled the machine learning training that object is emulated crawl at least once, grabbed with obtaining currently be crawled under substance environment optimal
Modulus shape parameter.
Further, the environment construction unit includes:
Model building module, for establishing each three-dimensional artificial for being crawled object and each handgrip respectively according to physical samples
Model, wherein the three-dimensional simulation model has multiple actual physical properties of the corresponding physical samples；
It is crawled environment and chooses module, established at random for choosing multiple three-dimensional simulation models for being crawled object and be crawled object
Environment；
Handgrip environment chooses module, is crawled object ring described in the three-dimensional simulation model correspondence of at least one handgrip for choosing
The handgrip environment is established in border setting；Wherein, the data for being crawled substance environment and the handgrip environment are mutually indepedent.
Further, the system also includes:
Incidence relation establishes unit, for establishing each three-dimensional simulation model for being crawled object according to physical samples, and leads to
The surface image that the three-dimensional simulation model analog sensor obtains is crossed, the surface image and the three-dimensional simulation model are established
And/or the incidence relation between the physical samples；
Model of mind training unit, for using the two dimensional image number of the three-dimensional simulation model, sensor acquisition
Accordingly and the incidence relation of foundation is trained neural network model, obtains for according to two-dimensional image data identification three
Tie up the first artificial intelligence model of object；Wherein, first artificial intelligence model can to two-dimensional image data to be identified into
Row identification, to obtain the corresponding three-dimensional information for being crawled object of the two-dimensional image data to be identified.
Further, the crawl model includes the second artificial intelligence model of planning crawl strategy and carries out handgrip control
Third artificial intelligence model；Wherein,
Second artificial intelligence model is planned according to the two-dimensional image data and the three-dimensional information for being crawled object
Export crawl task and implementation strategy；
The third artificial intelligence model controls the output of handgrip driving unit according to the implementation strategy, makes the handgrip
It grabs and is crawled object to realize the crawl task described in moving.
Further, when the method is applied to the Training scene of multiple handgrips, second artificial intelligence model
Output further includes handgrip selection strategy.
Further, the system also includes:
Real-time training unit, the fetching for being lower than preset threshold for such as can only obtain index in true environment, then
The three-dimensional information of true picture and the object obtained according to the true picture is inputted to online and is corresponded to the environment of building emulation,
The crawl model is trained in real time, and determines the fetching of the true environment by result trained in real time.
The another aspect of the embodiment of the present disclosure provides a kind of electronic equipment, comprising: at memory and one or more
Manage device；
Wherein, the memory is connect with one or more of processor communications, and being stored in the memory can quilt
The instruction that one or more of processors execute, when described instruction is executed by one or more of processors, the electronics
Equipment is for realizing the method as described in foregoing embodiments.
The another aspect of the embodiment of the present disclosure provides a kind of computer readable storage medium, and being stored thereon with computer can
It executes instruction, when the computer executable instructions are executed by a computing apparatus, can be used to realize as described in foregoing embodiments
Method.
The another further aspect of the embodiment of the present disclosure additionally provides a kind of computer program product, the computer program product packet
The computer program being stored on computer readable storage medium is included, the computer program includes program instruction, when the journey
When sequence instruction is computer-executed, it can be used to realize the method as described in foregoing embodiments.
Therefore technical solution provided by the present application, two dimensional image and entity mesh can establish by Computer Simulation
The prediction model between model is marked, and based on prediction model planning crawl strategy.This method is more suitable for the random heap of more objects
Folded scene, under the scene, gestures of object is more random, and there may be blocking between object, system obtains two by sensor
Image is tieed up, and the three-dimensional information of object can be predicted according to the two dimensional image.It, can be with after predicting the three-dimensional information of object
More accurately planning crawl strategy.The method proposed through the invention can complete a large amount of machinery within the very short time
Arm training operation, and then accelerate mechanical arm and be deployed to the period in different task.Especially, it does not need to waste a large amount of object
Material can find optimal fetching, such as this delicate articles of glass, and material object crawl failure to train means to make
At the loss of material, and loss then can be reduced to zero by the training of simulation type.
Detailed description of the invention
Fig. 1 is the flow chart of the mechanical gripper training method based on Computer Simulation in the embodiment of the present application；
Fig. 2 is the schematic diagram that various model construction simulated environment are used in one embodiment of the application；
Fig. 3 is the schematic diagram of a scenario for obtaining three-dimensional information in one embodiment of the application according to two dimensional image；
Fig. 4 is the schematic diagram of a scenario for carrying out simulation training in one embodiment of the application using more handgrips；
Fig. 5 is the implementation method flow diagram of simulation training in one embodiment of the application；
Fig. 6 is the functional block diagram of the mechanical gripper training system based on Computer Simulation in the embodiment of the present application.
Specific embodiment
In order to make those skilled in the art better understand the technical solutions in the application, below in conjunction with the application reality
The attached drawing in mode is applied, the technical solution in the application embodiment is clearly and completely described, it is clear that described
Embodiment is only a part of embodiment of the application, rather than whole embodiments.Based on the embodiment party in the application
Formula, all other embodiment obtained by those of ordinary skill in the art without making creative efforts, is all answered
When the range for belonging to the application protection.
In the prior art, the precise controlling of mechanical gripper needs complicated control logic to realize, and can only often be directed to
Fixed scene and task is pre-designed and configures, this makes the use scope of mechanical gripper limited and high expensive.Artificial intelligence
Although the appearance of energy technology can help the training of mechanical gripper to a certain extent, existing material object training method still can
It is limited by scene, time and frequency of training, efficiency and cost are unsatisfactory.Computer simulation technique can be brought further
Convenience, but existing simulation training directly uses the data parameters of simulation model, fails to the recognition capability to mechanical gripper
Effectively train, is also difficult to adapt to simultaneously for unbred new object new scene, there are still more apparent defects.This
Application provides a kind of mechanical gripper training method based on Computer Simulation, carries out strictly to the every aspect of crawl scene
Emulation and stand-alone training, so as to fast implement the training of mechanical gripper, and can be effectively applied to real scene.Specifically, it asks
Refering to fig. 1, which comprises
S1: what building emulated respectively is crawled substance environment and handgrip environment.
In the present embodiment, the substance environment and the handgrip environment of being crawled is all in accordance with the threedimensional model pre-established
Building, wherein the substance environment that is crawled generates at random and described is crawled substance environment and the handgrip environment is mutually indepedent.
Specifically, can be placed in the three-dimensional scenic of blank randomly select be crawled object three-dimensional model, selection it is more
A object three-dimensional model that is crawled arbitrarily is accumulated in certain space or is put to be formed and be crawled substance environment, and the three-dimensional mould of handgrip
The handgrip environment that type (including handgrip, sensor and handgrip control unit etc.) is constituted can be placed for substance environment is crawled.
Further, in order to obtain more fully training effect, simulation training is needed to carry out repeatedly, is also not limited to
It is carried out under same simulated environment；Therefore, emulation is crawled substance environment and handgrip environment and would generally repeatedly be weighed as needed
Multiple building, construct every time be crawled substance environment and/or handgrip environment can identical, part it is identical or entirely different etc.,
Herein should not building number to simulated environment and form make specific limitation.
In addition, in embodiments herein, the statement such as described " random to generate " and described " in certain space arbitrarily "
Each mean in simulation actual environment it can happen that, the case where for that can not exist in actual environment, for example do not meet
Physical rules are not inconsistent logical or do not meet the natural law, are not belonging to selection model when technical scheme is simulated
It encloses.
S2: using the two-dimensional image data being crawled described in the sensor acquisition in the handgrip environment in substance environment, and
The three-dimensional information for being crawled object is determined according to the two-dimensional image data.
In the present embodiment, the picture of imaging sensor shooting, can therefrom identify shape, the profile ruler of body surface
Very little, color, material etc., multiple objects stack when further identify stacked relation, mutual distance of object etc. (including handgrip with
It is crawled the relative position distance of object).Certainly can also be measured size by range sensor or laser radar etc., positional relationship and
The data such as distance.
Above-mentioned image recognition and determine three-dimensional information be by artificial intelligence technology, can be the people after training
Work model of mind regards black box as, inputs the two-dimensional image data to be identified of specified format, and model can export three-dimensional information automatically
Recognition result, the operation such as feature extraction, confidence calculations of multi-layer is voluntarily carried out inside model, is understood without user and dry
In advance.
S3: it according to the three-dimensional information for being crawled object, controls handgrip and carries out emulation crawl at least once to object is crawled
Machine learning training, and obtain and be currently crawled Optimal Grasp model parameter under substance environment.
In one embodiment of the application, different crawl strategies can be repeatedly attempted in simulated environment.That is, it is right
It is crawled substance environment and handgrip environment in the emulation that one has constructed, if crawl failure or crawl knot when certain simulated training
Fruit is undesirable, can reselect crawl strategy at any time by scenario reduction to original state or some intermediate state or part is adjusted
Simulated training again is carried out after whole, until obtaining optimal result.Due to using emulation mode, the ultrahigh in efficiency of scenario reduction
And cost is minimum, while will not be to being crawled object and/or handgrip causes any physical injury, thus can support to the maximum extent anti-
Refreshment is practiced, it is ensured that training obtains optimal result.
Wherein, the output that the Optimal Grasp model parameter expression grabs model when using the parameter, which can achieve, most to be managed
The result thought.The crawl model generally includes the second artificial intelligence model of planning crawl strategy and carries out the of handgrip control
Three artificial intelligence models；Wherein, second artificial intelligence model is according to the two-dimensional image data and the object that is crawled
Three-dimensional information planning output crawl task and implementation strategy；The third artificial intelligence model is grabbed according to implementation strategy control
The output of hand-drive unit grabs the handgrip and is crawled object to realize the crawl task described in moving.Thus more
Body, optimal result refer to that crawl model can be with according to the control amount of two-dimensional image data and three-dimensional information output in scene
Handgrip is set utmostly to meet the indices for completing crawl task.Wherein, the index that task is completed may include at least one
Performance requirement, for example success rate highest, grasp speed are most fast, single can grab that object is most, sustainable crawl number is most, most
It is energy saving, most quiet, most steady, failure rate is most low；It is highly preferred that weight that can also be certain to each setup measures, according to more
The comprehensive performance of a index is optimal to select.
Certainly, in addition to by the way of above-mentioned third artificial intelligence model, crawl strategy is converted to, list is driven to handgrip
The control signal of member also can be used a variety of feasible embodiments, for example use traditional drive control logic, by by plan
Slightly target is also able to achieve the drive control of handgrip to the conversion (when necessary further including digital-to-analogue conversion) of computer instruction.Therefore, on
State the limitation that the specific implementation to crawl strategy to handgrip control is not construed as in embodiment, any feasible drive control
Mode may be applicable in embodiments herein.
Therefore the application can be established pre- between two dimensional image and physical object model by Computer Simulation
Model is surveyed, and based on prediction model planning crawl strategy, is more suitable for the scene that more objects stack at random, at this time gestures of object
More random, there may be blocking between object, system obtains two dimensional image by imaging sensor, but corresponding by predicting
Target object, more accurately planning crawl is tactful.
Some preferred embodiments of the application are described in detail in conjunction with each schematic diagram further below.As shown in Fig. 2, at this
Application a preferred embodiment in, emulation be crawled substance environment and handgrip environment can be respectively by the three of multiple target objects
Dimension module is composed.For example, being crawled in substance environment, it may include the model 211 of various materials, randomly select any amount
Material model 211 be deposited in certain space naturally；Correspondingly, can also be arranged in being crawled substance environment some relevant
Accessory model 212, such as charging tray, shelf, conveyer belt, configuration and usage mode are identical as real-world object, herein no longer one by one
Expansion description.And in handgrip environment, also may include the model 221 of various handgrips, typically as sucked type, mechanical gripping finger formula,
Flexible finger formula etc.；Handgrip model 221 may be mounted on executing agency (such as mechanical arm, robot) model 223, simultaneously also
It is provided with camera model 222, for simulating the image acquisition process of true handgrip.
In an application example, the concrete methods of realizing of simulated environment are as follows: the object of target object is established by computer
Manage model, including center of gravity, surface roughness, shape etc..Establish the scene that multiple target objects stack.Establish the physics mould of handgrip
Type.The two dimensional image of the physics scene is obtained, planning crawl strategy goes test crawl success rate by handgrip model.Based on object
Manage the crawl success rate of emulation, training crawl strategic planning model.
As shown in figure 3, being crawled the three of object according to two-dimensional image data determination in the preferred embodiment of the application
Dimension information can divide multiple steps to realize.In Fig. 3, it is crawled first by the whole that camera model acquisition is crawled in substance environment
The two dimensional image of object (two dimensional image on the top Fig. 3 is using top view as example)；Then the object in two dimensional image is identified,
Obtain the relative positional relationship of multiple objects and the surface image data of accumulation body (left under Fig. 3 is a lateral plan example)；
Further according to all images data for the threedimensional model completion accumulation body for identifying object, (what especially lower part was blocked can not
See the data of part；Fig. 3 lower right is the example of same side view).It can be accumulated by all images data of completion
The Arbitrary 3 D information of each different angle of body, to may be selected that the grasp mode for being respectively crawled object (handgrip choosing can be efficiently separated
Select, grab angle Selection, grasping force selection, direction of motion selection etc.), carry out accurate crawl strategic planning.
Wherein, infer that the three-dimensional information for being crawled object can also be realized by machine learning mode according to two-dimensional image data,
The threedimensional model of target object, and the surface image that analog image sensor obtains are established in physical simulation engine, are obtained
Relationship between the two；Neural network model is trained using threedimensional model, image and both sides relation, can be obtained by two dimension
The artificial intelligence model of image data identification three-dimension object.
Computer Simulation training can also train more handgrips.I.e. grasping system includes multiple handgrips, and multiple handgrips have not
Same feature, such as software, hardware, surface roughness etc..Also multiple handgrips are simulated in handgrip environment respectively to being crawled object ring
Border carries out crawl training.Handgrip selection method and fetching planing method of the training grasping system to a task.Specifically,
Assessment system serially or simultaneously can allow a variety of handgrips to execute identical task, every kind of handgrip is individually according to current task training
Corresponding grasping means, when every kind of grasping means is optimal, the performance of more a variety of handgrips obtains optimal handgrip selecting party
Case.
Further as shown in figure 4, optimal handgrip selection scheme can also be more in the preferred embodiment of the application
Kind handgrip is applied in combination, and selects different handgrips to realize such as multiple concrete conditions for being crawled object and grabs respectively.?
In Fig. 4, handgrip model includes the diversified forms such as sucked type, mechanical gripping finger formula, flexible finger formula, according to being crawled in substance environment
Multiple recognition results for being crawled object, planning be crawled object crawl sequence and every time crawl when the handgrip type that uses.Allusion quotation
Type, in Fig. 4, first using the cube of sucked type handgrip crawl accumulation body most surface, then grabbed using gripping finger formula handgrip
The L shape object for taking left side edge not interfered by other objects reuses the flexible finger formula handgrip crawl current most surface of accumulation body
Sphere, then using sucked type handgrip crawl trigone cone etc..Using above-mentioned preferred embodiment, the application can be by more
Kind handgrip realizes most preferred crawl strategic planning.
In practical applications, the scheme of the application can be realized according to the following steps:
A: by physical simulation engine, fine modeling, including contoured article, color, matter are carried out to the article of required crawl
Amount, center of gravity, surface roughness, elasticity modulus etc..
B: modeling fixture (including sucker), in addition to essential characteristics such as the shape of fixture, sizes, should also establish it and grab
The assessment algorithm of article is taken, the lifting force etc. that for example vacuum can be generated after sucker contact object is various more realistically to emulate
Can fixture effectively grab article in varied situations.
C: crawl task modeling, including the carrying of the move loading action, mechanical arm system used, article to be realized or transmission dress
It sets, article supplied materials form (stacking, tiling or placement at random), i.e., in existing " article " and " fixture " the two key elements
After refined model, other simulated environment are established by true application scenarios.
Building mechanical arm, fixture, camera, material and the dummy model and gravity of associated satellite equipment, environmental parameter are complete
Complete simulated environment.Wherein material model is other than category is complete, wherein the dummy model includes the model of physical attribute, example
Such as quality, density, center of gravity, surface characteristic；In order to which real simulation grabs effect in a simulated environment.
Herein, simulated environment modeling is more complete, and it is more convenient that training result is transferred to true environment, without resetting
The variables such as pick-and-place position, actuating range constraint.
It (for example grabs success rate after completing training in simulated environment and reaches 99.9%), should also be tested in true environment
Card or amendment, intensified learning.
D: simulation carries out crawl operation, i.e. virtual training in virtual environment, obtains the optimal grasping algorithm of crawl effect.
Optionally, simulation engine can receive artificial mark, such as different articles in Optimal Grasp in different positions
Scheme is as emulation reference.In addition algorithm can also be carried out at training initial stage by way of manual intervention (calibration, guidance)
Guidance, so that simulation result is restrained as early as possible.
With further reference to Fig. 5, in the preferred embodiment of the application, complete simulation training can be multiple singles
Grab the circulation of simulation training process.Start epicycle simulation training, random product material heap after initial construction simulated environment
Cumuliformis state, the crawl training then gradually emulated.Primary phase can be considered as the training of each grasping movement
Independent single is grabbed and is emulated, as shown on the right side of Fig. 5, single crawl emulation from scanning surface of material generate two dimensional image, by
Two dimensional image matching material data obtains three-dimensional information and starts, and inside further may include one and grab test/training repeatedly
Cyclic process: for grabbing trial every time, the target preferentially grabbed, suitable fixture is first selected and properly (Fig. 5 only makees crawl posture
For a kind of example, the sequences of several selections can not be carried out according to the example of Fig. 5), then control is implemented emulation and is grabbed, according to grabbing
Result is taken to judge effect (whether grab successfully, time-consuming, object state etc. can be used as judgment criteria) is attempted in this crawl, then
It updates crawl Policy model and/or selection is attempted again；If attempted again, material state enters before restoring crawl
Circulation, change strategy carry out emulation crawl again.After implementing single crawl emulation, further determine whether to complete epicycle training mesh
Mark (such as all crawl completions etc.) recycles if not completing training objective and carries out single crawl emulation；If completing training objective
It then further determines whether to need to carry out re -training (such as new scene, new object, new handgrip etc.), if you need to weight
New training, then product material stacking states, circulation carry out single crawl emulation again；If judgement is not necessarily to re -training, then export
Current training pattern terminates simulation training.
Certainly, relevant technical staff in the field is appreciated that the process description of Fig. 5 property as an example, can be used and appoints
Feasible computer system of anticipating realizes that execution sequence is not necessarily to strictly carry out according to the example of Fig. 5, and can handle energy according to system
Power and/or scene need and the execution sequence of any set-up procedure or take parallel processing to certain steps, therefore Fig. 5 is not construed as
Concrete restriction made by method flow to the embodiment of the present application.
Above method further includes two kinds of application modes:
Off-line training: Computer Simulation training is carried out in a manner of offline pre-training, i.e., is carried out by virtual simulation environment big
The simulation of amount disposes grasping algorithm after obtaining enough analogue datas and completing fetching training in true environment,
Capture apparatus directly generates fetching using the model that pre-training goes out according to the true picture of acquisition in true environment.
Training in real time: pre-training mode is same as above, but is such as encountered in true environment and be difficult to provide the higher crawl of success rate
When scheme (such as when encountering the environment or object without training up), by true picture and according to image obtain object three
Dimension information is input in virtual simulation environment online, is trained in real time to crawl model, and determine by result trained in real time
Fetching completes crawl.
The application scheme provided above, training can be divided into the machine learning for having supervision and unsupervised machine learning,
Difference is in the data inputted whether include mark to result.Any training method may be applicable to the scheme of the application,
But supervised learning mode more preferably is taken to prediction model, unsupervised learning mode is taken to Grasp Planning model.
The input of prediction model is the two dimensional image for being crawled object, export for be crawled the substance parameter of object (such as including
Position, shape, material, surface roughness, quality, center of gravity etc.)；The input of Grasp Planning model is above-mentioned substance parameter, output
For handgrip crawl strategy (such as including handgrip type, the direction of motion, speed, crawl angle, grasping force, extraction rate, mention
Take height etc.).
Two models are required to by training in advance, and the scheme of the application can be divided into two processes, first is that each model
Training process, second is that using training after model help handgrip complete grasping movement, core is training process, and key point is
The application is that virtual training is carried out in simulated environment.
It blocks, can be predicted as far as possible according to the data that currently would know that, and grab in principle if grabbed object and existed
Be object is caught away one by one from top to down, therefore have block on a small quantity it is little to predicted impact, when serious shielding can first grab or
Adjust shelter.Specific grasp mode is voluntarily judged that prediction model only needs to export as far as possible here quasi- by Grasp Planning model
True prediction result can (exist seriously block be also prediction result one kind).
The relationship of three-dimensional stereo model and two-dimensional surface image can be the corresponding mark in respective data；It can be by artificial
Addition can also be added automatically in simulation.
It is trained below using three-dimensional model, image and relationship as the machine learning training process for having supervision, wherein
Image is the sample of input, is mark according to the three-dimensional model of Relation acquisition, and neural network carries out feature extraction to sample, with examination
Figure is established metastable confidence level between sample and mark and is contacted.
More handgrips refer to handgrip type can there are many, every kind of handgrip is respectively trained, be directed to different crawl tasks
When can choose the handgrip type of most suitable (success rate is most high) to operate.The handgrip quantity of control with no restrictions, passes simultaneously
System automatic control technology can be realized while control multiple components to execute different operation.
Every kind of handgrip is respectively trained, and selects after the completion of each self-training according still further to certain index comprehensive optimal such as successful
Rate highest, grasp speed are most fast, single can grab that object is most, sustainable crawl number is most, most energy saving, most quiet, most flat
Surely, failure rate is most low.
Referring to Fig. 6, the application also provides a kind of mechanical gripper training system based on Computer Simulation, the system packet
It includes:
Environment construction unit is crawled substance environment and handgrip environment for construct emulation respectively；
Simulation unit, for utilizing two be crawled in substance environment described in the sensor Simulation acquisition in the handgrip environment
Dimensional data image, and determine that at least one is crawled the three-dimensional information of object according to the two-dimensional image data；
Training unit controls the handgrip in the handgrip environment to institute for being crawled the three-dimensional information of object according to
It states and is crawled the machine learning training that object is emulated crawl at least once, grabbed with obtaining currently be crawled under substance environment optimal
Modulus shape parameter.
In one embodiment, the environment construction unit includes:
Model building module, for establishing each three-dimensional artificial for being crawled object and each handgrip respectively according to physical samples
Model, wherein the three-dimensional simulation model has multiple actual physical properties of the corresponding physical samples；
It is crawled environment and chooses module, established at random for choosing multiple three-dimensional simulation models for being crawled object and be crawled object
Environment；
Handgrip environment chooses module, is crawled object ring described in the three-dimensional simulation model correspondence of at least one handgrip for choosing
The handgrip environment is established in border setting；Wherein, the data for being crawled substance environment and the handgrip environment are mutually indepedent.
In one embodiment, the system also includes:
Incidence relation establishes unit, for establishing each three-dimensional simulation model for being crawled object according to physical samples, and leads to
The surface image that the three-dimensional simulation model analog sensor obtains is crossed, the surface image and the three-dimensional simulation model are established
And/or the incidence relation between the physical samples；
Model of mind training unit, for using the two dimensional image number of the three-dimensional simulation model, sensor acquisition
Accordingly and the incidence relation of foundation is trained neural network model, obtains for according to two-dimensional image data identification three
Tie up the first artificial intelligence model of object；Wherein, first artificial intelligence model can to two-dimensional image data to be identified into
Row identification, to obtain the corresponding three-dimensional information for being crawled object of the two-dimensional image data to be identified.
In one embodiment, the crawl model includes the second artificial intelligence model and the progress of planning crawl strategy
The third artificial intelligence model of handgrip control；Wherein,
Second artificial intelligence model is planned according to the two-dimensional image data and the three-dimensional information for being crawled object
Export crawl task and implementation strategy；
The third artificial intelligence model controls the output of handgrip driving unit according to the implementation strategy, makes the handgrip
It grabs and is crawled object to realize the crawl task described in moving.
In one embodiment, when the method is applied to the Training scene of multiple handgrips, the second artificial intelligence
The output of energy model further includes handgrip selection strategy.
In one embodiment, the system also includes:
Real-time training unit, the fetching for being lower than preset threshold for such as can only obtain index in true environment, then
The three-dimensional information of true picture and the object obtained according to the true picture is inputted to online and is corresponded to the environment of building emulation,
The crawl model is trained in real time, and determines the fetching of the true environment by result trained in real time.
Therefore technical solution provided by the present application, two dimensional image and entity mesh can establish by Computer Simulation
The prediction model between model is marked, and based on prediction model planning crawl strategy.This method is more suitable for the random heap of more objects
Folded scene, under the scene, gestures of object is more random, and there may be blocking between object, system obtains two by sensor
Image is tieed up, and the three-dimensional information of object can be predicted according to the two dimensional image.It, can be with after predicting the three-dimensional information of object
More accurately planning crawl strategy.The method proposed through the invention can complete a large amount of machinery within the very short time
Arm training operation, and then accelerate mechanical arm and be deployed to the period in different task.Especially, it does not need to waste a large amount of object
Material can find optimal fetching, such as this delicate articles of glass, and material object crawl failure to train means to make
At the loss of material, and loss then can be reduced to zero by the training of simulation type.
Those skilled in the art are supplied to the purpose described to the description of the various embodiments of the application above.It is not
It is intended to exhaustion or be not intended to and limit the invention to single disclosed embodiment.As described above, the application's is various
Substitution and variation will be apparent for above-mentioned technology one of ordinary skill in the art.Therefore, although specifically begging for
Some alternative embodiments are discussed, but other embodiment will be apparent or those skilled in the art are opposite
It is easy to obtain.The application is intended to include all substitutions of the invention discussed herein, modification and variation, and falls in
Other embodiment in the spirit and scope of above-mentioned application.
Claims (12)
1. a kind of mechanical gripper training method based on Computer Simulation, which is characterized in that the described method includes:
What building emulated respectively is crawled substance environment and handgrip environment；
Using the two-dimensional image data being crawled described in the sensor Simulation acquisition in the handgrip environment in substance environment, and according to
The two-dimensional image data determines that at least one is crawled the three-dimensional information of object；
According to the three-dimensional information for being crawled object, the handgrip controlled in the handgrip environment carries out at least the object that is crawled
The machine learning training of primary emulation crawl, to obtain the Optimal Grasp model parameter being currently crawled under substance environment.
2. the method according to claim 1, wherein the emulation of building respectively is crawled substance environment and handgrip
Environment includes:
Establish each three-dimensional simulation model for being crawled object and each handgrip respectively according to physical samples, wherein described three-dimensional imitative
True mode has multiple actual physical properties of the corresponding physical samples；
It chooses multiple three-dimensional simulation models for being crawled object and establishes at random and be crawled substance environment；
Choose at least one handgrip three-dimensional simulation model it is corresponding described in be crawled substance environment setting and establish the handgrip environment；Its
In, the data for being crawled substance environment and the handgrip environment are mutually indepedent.
3. the method according to claim 1, wherein the method also includes:
Each three-dimensional simulation model for being crawled object is established according to physical samples, and passes through the three-dimensional simulation model analog sensed
The surface image that device obtains establishes the pass between the surface image and the three-dimensional simulation model and/or the physical samples
Connection relationship；
Use the incidence relation pair of the three-dimensional simulation model, the two-dimensional image data that the sensor acquires and foundation
Neural network model is trained, and obtains the first artificial intelligence model for identifying three-dimension object according to two-dimensional image data；
Wherein, first artificial intelligence model can identify two-dimensional image data to be identified, described to be identified to obtain
The corresponding three-dimensional information for being crawled object of two-dimensional image data.
4. the method according to claim 1, wherein the crawl model includes the second people of planning crawl strategy
Work model of mind and the third artificial intelligence model for carrying out handgrip control；Wherein,
Second artificial intelligence model is according to the two-dimensional image data and the three-dimensional information planning output for being crawled object
Crawl task and implementation strategy；
The third artificial intelligence model controls the output of handgrip driving unit according to the implementation strategy, grabs the handgrip
And object is crawled to realize the crawl task described in movement.
5. according to the method described in claim 4, it is characterized in that, when the method is applied to the Training scene of multiple handgrips
When, the output of second artificial intelligence model further includes handgrip selection strategy.
6. the method according to claim 1, wherein the method also includes:
It such as can only obtain the fetching that index is lower than preset threshold in true environment, then by true picture and according to described true
The three-dimensional information for the object that real image obtains inputs online and the environment of corresponding building emulation, carries out to the crawl model real-time
It trains, and determines the fetching of the true environment by result trained in real time.
7. a kind of mechanical gripper training system based on Computer Simulation, which is characterized in that the system comprises:
Environment construction unit is crawled substance environment and handgrip environment for construct emulation respectively；
Simulation unit, for utilizing the X-Y scheme being crawled in substance environment described in the sensor Simulation acquisition in the handgrip environment
As data, and determine that at least one is crawled the three-dimensional information of object according to the two-dimensional image data；
Training unit controls the handgrip in the handgrip environment to the quilt for being crawled the three-dimensional information of object according to
Crawl object is emulated the machine learning training of crawl at least once, to obtain the Optimal Grasp mould being currently crawled under substance environment
Shape parameter.
8. system according to claim 7, which is characterized in that the environment construction unit includes:
Model building module, for establishing each three-dimensional artificial mould for being crawled object and each handgrip respectively according to physical samples
Type, wherein the three-dimensional simulation model has multiple actual physical properties of the corresponding physical samples；
It is crawled environment and chooses module, established at random for choosing multiple three-dimensional simulation models for being crawled object and be crawled object ring
Border；
Handgrip environment chooses module, and being crawled substance environment described in the three-dimensional simulation model for choosing at least one handgrip is corresponding sets
It sets up and founds the handgrip environment；Wherein, the data for being crawled substance environment and the handgrip environment are mutually indepedent.
9. system according to claim 7, which is characterized in that the system also includes:
Incidence relation establishes unit, for establishing each three-dimensional simulation model for being crawled object according to physical samples, and passes through institute
State the surface image that three-dimensional simulation model analog sensor obtains, establish the surface image and the three-dimensional simulation model and/
Or the incidence relation between the physical samples；
Model of mind training unit, for use the three-dimensional simulation model, the sensor acquisition two-dimensional image data with
And the incidence relation established is trained neural network model, obtains for identifying three-dimensional article according to two-dimensional image data
First artificial intelligence model of body；Wherein, first artificial intelligence model can know two-dimensional image data to be identified
Not, to obtain the corresponding three-dimensional information for being crawled object of the two-dimensional image data to be identified.
10. system according to claim 7, which is characterized in that the crawl model includes the second of planning crawl strategy
Artificial intelligence model and the third artificial intelligence model for carrying out handgrip control；Wherein,
Second artificial intelligence model is according to the two-dimensional image data and the three-dimensional information planning output for being crawled object
Crawl task and implementation strategy；
The third artificial intelligence model controls the output of handgrip driving unit according to the implementation strategy, grabs the handgrip
And object is crawled to realize the crawl task described in movement.
11. system according to claim 10, which is characterized in that when the method is applied to the Training scene of multiple handgrips
When, the output of second artificial intelligence model further includes handgrip selection strategy.
12. system according to claim 7, which is characterized in that the system also includes:
Real-time training unit, the fetching for being lower than preset threshold for such as can only obtain index in true environment, then will be true
The three-dimensional information of real image and the object obtained according to the true picture inputs online and the environment of corresponding building emulation, to institute
It states crawl model to be trained in real time, and determines the fetching of the true environment by result trained in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910398732.3A CN110216671A (en) | 2019-05-14 | 2019-05-14 | A kind of mechanical gripper training method and system based on Computer Simulation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910398732.3A CN110216671A (en) | 2019-05-14 | 2019-05-14 | A kind of mechanical gripper training method and system based on Computer Simulation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110216671A true CN110216671A (en) | 2019-09-10 |
Family
ID=67821069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910398732.3A Pending CN110216671A (en) | 2019-05-14 | 2019-05-14 | A kind of mechanical gripper training method and system based on Computer Simulation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110216671A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110948492A (en) * | 2019-12-23 | 2020-04-03 | 浙江大学 | Three-dimensional grabbing platform and grabbing method based on deep learning |
| CN111730606A (en) * | 2020-08-13 | 2020-10-02 | 深圳国信泰富科技有限公司 | Grabbing action control method and system of high-intelligence robot |
| CN112435521A (en) * | 2020-10-27 | 2021-03-02 | 北京控制工程研究所 | Space teleoperation mechanical arm on-orbit training system and method |
| US20210122038A1 (en) * | 2018-06-28 | 2021-04-29 | Siemens Aktiengesellschaft | Method and device for ascertaining control parameters in a computer-assisted manner for a favorable action of a technical system |
| CN113034668A (en) * | 2021-03-01 | 2021-06-25 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
| EP3944934A1 (en) * | 2020-07-29 | 2022-02-02 | Liebherr-Verzahntechnik GmbH | Method for automatically determining the construction parameters of a gripper |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938668A (en) * | 2010-09-10 | 2011-01-05 | 中国科学院自动化研究所 | Method for three-dimensional reconstruction of multilevel lens multi-view scene |
| CN102663814A (en) * | 2012-03-30 | 2012-09-12 | 上海第二工业大学 | Automatic modeling method utilizing two dimensional image to generate three dimensional geometric model |
| CN102682477A (en) * | 2012-05-16 | 2012-09-19 | 南京邮电大学 | Regular scene three-dimensional information extracting method based on structure prior |
| EP2500148A1 (en) * | 2011-03-17 | 2012-09-19 | KUKA Roboter GmbH | Method and apparatus for controlling a robot using a virtual model of the robot |
| CN103759670A (en) * | 2014-01-06 | 2014-04-30 | 四川虹微技术有限公司 | Object three-dimensional information acquisition method based on digital close range photography |
| CN105809734A (en) * | 2016-03-10 | 2016-07-27 | 杭州师范大学 | Mechanical model 3D modeling method based on multiple view interaction |
| CN108058172A (en) * | 2017-11-30 | 2018-05-22 | 深圳市唯特视科技有限公司 | A kind of manipulator grasping means based on autoregression model |
| CN108972494A (en) * | 2018-06-22 | 2018-12-11 | 华南理工大学 | A kind of Apery manipulator crawl control system and its data processing method |
| CN109145766A (en) * | 2018-07-27 | 2019-01-04 | 北京旷视科技有限公司 | Model training method, device, recognition methods, electronic equipment and storage medium |
| CN109483534A (en) * | 2018-11-08 | 2019-03-19 | 腾讯科技（深圳）有限公司 | A kind of grasping body methods, devices and systems |
-
2019
- 2019-05-14 CN CN201910398732.3A patent/CN110216671A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938668A (en) * | 2010-09-10 | 2011-01-05 | 中国科学院自动化研究所 | Method for three-dimensional reconstruction of multilevel lens multi-view scene |
| EP2500148A1 (en) * | 2011-03-17 | 2012-09-19 | KUKA Roboter GmbH | Method and apparatus for controlling a robot using a virtual model of the robot |
| CN102663814A (en) * | 2012-03-30 | 2012-09-12 | 上海第二工业大学 | Automatic modeling method utilizing two dimensional image to generate three dimensional geometric model |
| CN102682477A (en) * | 2012-05-16 | 2012-09-19 | 南京邮电大学 | Regular scene three-dimensional information extracting method based on structure prior |
| CN103759670A (en) * | 2014-01-06 | 2014-04-30 | 四川虹微技术有限公司 | Object three-dimensional information acquisition method based on digital close range photography |
| CN105809734A (en) * | 2016-03-10 | 2016-07-27 | 杭州师范大学 | Mechanical model 3D modeling method based on multiple view interaction |
| CN108058172A (en) * | 2017-11-30 | 2018-05-22 | 深圳市唯特视科技有限公司 | A kind of manipulator grasping means based on autoregression model |
| CN108972494A (en) * | 2018-06-22 | 2018-12-11 | 华南理工大学 | A kind of Apery manipulator crawl control system and its data processing method |
| CN109145766A (en) * | 2018-07-27 | 2019-01-04 | 北京旷视科技有限公司 | Model training method, device, recognition methods, electronic equipment and storage medium |
| CN109483534A (en) * | 2018-11-08 | 2019-03-19 | 腾讯科技（深圳）有限公司 | A kind of grasping body methods, devices and systems |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210122038A1 (en) * | 2018-06-28 | 2021-04-29 | Siemens Aktiengesellschaft | Method and device for ascertaining control parameters in a computer-assisted manner for a favorable action of a technical system |
| CN110948492A (en) * | 2019-12-23 | 2020-04-03 | 浙江大学 | Three-dimensional grabbing platform and grabbing method based on deep learning |
| CN110948492B (en) * | 2019-12-23 | 2021-10-22 | 浙江大学 | Three-dimensional grabbing platform and grabbing method based on deep learning |
| EP3944934A1 (en) * | 2020-07-29 | 2022-02-02 | Liebherr-Verzahntechnik GmbH | Method for automatically determining the construction parameters of a gripper |
| CN111730606A (en) * | 2020-08-13 | 2020-10-02 | 深圳国信泰富科技有限公司 | Grabbing action control method and system of high-intelligence robot |
| CN111730606B (en) * | 2020-08-13 | 2022-03-04 | 深圳国信泰富科技有限公司 | Grabbing action control method and system of high-intelligence robot |
| CN112435521A (en) * | 2020-10-27 | 2021-03-02 | 北京控制工程研究所 | Space teleoperation mechanical arm on-orbit training system and method |
| CN113034668A (en) * | 2021-03-01 | 2021-06-25 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
| CN113034668B (en) * | 2021-03-01 | 2023-04-07 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110216671A (en) | A kind of mechanical gripper training method and system based on Computer Simulation | |
| James et al. | Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task | |
| CN109483573A (en) | Machine learning device, robot system and machine learning method | |
| Ganapathi et al. | Learning dense visual correspondences in simulation to smooth and fold real fabrics | |
| Mariolis et al. | Pose and category recognition of highly deformable objects using deep learning | |
| JP6671694B1 (en) | Machine learning device, machine learning system, data processing system, and machine learning method | |
| CN110400345B (en) | Deep reinforcement learning-based radioactive waste push-grab cooperative sorting method | |
| Bousmalis et al. | Robocat: A self-improving foundation agent for robotic manipulation | |
| CN110125930A (en) | It is a kind of that control method is grabbed based on the mechanical arm of machine vision and deep learning | |
| Jørgensen et al. | An adaptive robotic system for doing pick and place operations with deformable objects | |
| Ganapathi et al. | Learning to smooth and fold real fabric using dense object descriptors trained on synthetic color images | |
| CN109483534A (en) | A kind of grasping body methods, devices and systems | |
| Sun et al. | Learning pregrasp manipulation of objects from ungraspable poses | |
| CN110152290A (en) | Game running method and device, storage medium and electronic device | |
| CN113076615B (en) | High-robustness mechanical arm operation method and system based on antagonistic deep reinforcement learning | |
| Mahjourian et al. | Hierarchical policy design for sample-efficient learning of robot table tennis through self-play | |
| CN110303498A (en) | Handling system and its control method, floor tile paving system | |
| Demura et al. | Picking up one of the folded and stacked towels by a single arm robot | |
| Kołosowski et al. | Collaborative robot system for playing chess | |
| JP2022187983A (en) | Network modularization to learn high dimensional robot tasks | |
| Bahety et al. | Bag all you need: Learning a generalizable bagging strategy for heterogeneous objects | |
| Zhang et al. | Reinforcement learning based pushing and grasping objects from ungraspable poses | |
| CN113436293B (en) | Intelligent captured image generation method based on condition generation type countermeasure network | |
| Biza et al. | One-shot imitation learning via interaction warping | |
| JP2022187984A (en) | Grasping device using modularized neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190910 |
|
| RJ01 | Rejection of invention patent application after publication |