CN109909998A

CN109909998A - A kind of method and device controlling manipulator motion

Info

Publication number: CN109909998A
Application number: CN201711320833.6A
Authority: CN
Inventors: 马星辰
Original assignee: Beijing Orion Star Technology Co Ltd
Current assignee: Beijing Orion Star Technology Co Ltd
Priority date: 2017-12-12
Filing date: 2017-12-12
Publication date: 2019-06-21
Anticipated expiration: 2037-12-12
Also published as: CN109909998B

Abstract

The embodiment of the invention provides a kind of method and devices for controlling manipulator motion, the described method includes: control equipment control mechanical arm is according to the mobile pre-determined distance in predetermined movement direction, and camera is obtained in the target image of the target object of the mobile when institute continuous collecting of the mechanical arm, and acquire the location information of mechanical arm when every target image, feature extraction is carried out to target image, obtain the characteristics of image of target image, according to the corresponding relationship of preset characteristics of image and label, obtain the corresponding label of each target image, then judge in the corresponding label of each target image, with the presence or absence of the label for meeting preset condition, if there is, the target position information of mechanical arm when determining the existing label of acquisition corresponding target image, control mechanical arm is moved to the corresponding position of target position information.As a result, by way of two secondary control manipulator motions, mechanical arm can be made to be moved to predeterminated position, reduce the time of control manipulator motion.

Description

A kind of method and device controlling manipulator motion

Technical field

The present invention relates to mechanical arm control fields, more particularly to a kind of method and device for controlling manipulator motion.

Background technique

Currently, the application scenarios using mechanical arm are many, for example, automatic sorting cargo, automatic assembling components etc.. During using mechanical arm, need to control manipulator motion.

The process of control manipulator motion is general at present are as follows: control equipment obtains the camera acquisition installed on mechanical arm The image of target object, by the image be input in advance training complete convolutional neural networks in, the convolutional neural networks according to The position of objects in images and the current position of mechanical arm, movement that calculating machine arm needs to be implemented in next step (such as to the left Mobile or crawl) it is exported, control equipment can control mechanical arm and execute the movement, then repeats the above process, until It is moved to predeterminated position.

As it can be seen that the method for above-mentioned control manipulator motion has the following disadvantages: that the scheme of control manipulator motion is basis The movement of next step is calculated in mechanical arm current state, manipulator motion is controlled, then further according to the mechanical arm after movement Current state, then the movement of next step is calculated, then control manipulator motion.That is control equipment needs repeatedly control machine The movement of tool arm just reaches predeterminated position, and since each calculating action needs the time, mechanical arm execution movement is also required to the time, because This, control equipment repeatedly controls manipulator motion and needs to expend longer time, such as: assuming that calculating action spends time t1, Mechanical arm execution movement spends time t2, and mechanical arm moves n step altogether, then the process for controlling manipulator motion spends altogether time n (t1+t2), in general n and t2 are larger, and therefore, the process is more time-consuming.

Summary of the invention

The embodiment of the present invention is designed to provide a kind of method and device for controlling manipulator motion, to reduce control machine The time of tool arm movement.Specific technical solution is as follows:

A method of control manipulator motion, applied to the control equipment communicated to connect with the mechanical arm, the control Control equipment is also communicated to connect with camera, which comprises

The mechanical arm is controlled according to the mobile pre-determined distance in predetermined movement direction, and obtains the camera in the machinery The target image of the target object of the mobile when institute continuous collecting of arm, and acquire the position of mechanical arm when every target image Information；

Feature extraction is carried out to the target image, the characteristics of image of the target image is obtained, according to preset image The corresponding relationship of feature and label obtains the corresponding label of each target image, wherein the label is for identifying the machinery The relative positional relationship of arm and the target object；

Judge in the corresponding label of each target image, if there is the label for meeting preset condition；

If there is the label for meeting preset condition, machine when the existing label of acquisition corresponding target image is determined The target position information of tool arm；

It controls the mechanical arm and is moved to the corresponding position of the target position information.

Optionally, the process in the predetermined movement direction is obtained, comprising:

Obtain the present image at the target object current time of the camera acquisition；

Feature extraction is carried out to the present image, the characteristics of image of the present image is obtained, according to preset image The corresponding relationship of feature and label obtains the corresponding label of the present image；

According to the corresponding relationship of preset label and the direction of motion, determine corresponding to the corresponding label of the present image Predetermined movement direction.

Optionally, the method also includes:

If not there is no the label for meeting preset condition, returns and execute the control mechanical arm according to predetermined movement side To the step of moving pre-determined distance.

Optionally, described that feature extraction is carried out to the target image, the characteristics of image of the target image is obtained, according to The corresponding relationship of preset characteristics of image and label, the step of obtaining each target image corresponding label, comprising:

The target image is inputted into the target convolutional neural networks that training is completed in advance, so that the target convolutional Neural Network carries out feature extraction to the target image, the characteristics of image of the target image is obtained, according to self-contained image The characteristics of image of sample and the corresponding relationship of label obtain the corresponding label of each target image；

Wherein, the target convolutional neural networks are as follows: it is based on image pattern and its corresponding label, it is first to what is constructed in advance Beginning convolutional neural networks are trained obtained convolutional neural networks, and the target convolutional neural networks include described image sample The corresponding relationship of characteristics of image originally and label.

Optionally, the training method of the target convolutional neural networks, comprising:

Construct initial convolutional neural networks；

The target object is placed in predeterminated position, changes the direction of motion of the mechanical arm, obtains the camera In the mechanical arm according to the multiple images sample of the target object of the mobile when institute continuous collecting of each direction of motion；

When according to acquiring each image pattern, the location information of the location information of the mechanical arm and the target object, According to default label create-rule, the corresponding label of each image pattern is determined；

Described image sample and its corresponding label are inputted the initial convolutional neural networks to be trained；

When the value of the objective function of the initial convolutional neural networks no longer changes or the corresponding output of described image sample As a result when accuracy rate reaches default accuracy rate, training is completed, obtains the characteristics of image and mark comprising described image sample The target convolutional neural networks of the corresponding relationship of label.

Optionally, the step of the corresponding position of the target mechanical arm location information is moved in the control mechanical arm After rapid, the method also includes:

Whether the existing label of judgement is identical as default end-tag；

If so, controlling the mechanical arm grabs the target object；

Optionally, when the mechanical arm grabs target object success, the method also includes:

Output crawl successful information.

A kind of device controlling manipulator motion, applied to the control equipment communicated to connect with the mechanical arm, the control Control equipment is also communicated to connect with camera, and described device includes:

Mechanical arm mobile module for controlling the mechanical arm according to the mobile pre-determined distance in predetermined movement direction, and obtains The camera moves the target image of the target object of when institute continuous collecting, and every target figure of acquisition in the mechanical arm As when the mechanical arm location information；

Label determining module, for carrying out feature extraction to the target image, the image for obtaining the target image is special Sign, according to the corresponding relationship of preset characteristics of image and label, obtains the corresponding label of each target image, wherein the mark Sign the relative positional relationship for identifying the mechanical arm Yu the target object；

First judgment module, for judging in the corresponding label of each target image, if there is the default item of satisfaction The label of part, if so, triggering target position information determining module；

The target position information determining module, when for determining the corresponding target image of the existing label of acquisition described in The target position information of mechanical arm；

Control module is moved to the corresponding position of the target position information for controlling the mechanical arm.

Optionally, described device further includes obtaining module, and the acquisition module is for obtaining the predetermined movement direction, institute State acquisition module, comprising:

Present image acquiring unit, the current figure at the target object current time for obtaining the camera acquisition Picture；

Tag determination unit, for carrying out feature extraction to the present image, the image for obtaining the present image is special Sign, according to the corresponding relationship of preset characteristics of image and label, obtains the corresponding label of the present image；

Predetermined movement direction-determining unit, for the corresponding relationship according to preset label and the direction of motion, determine described in Predetermined movement direction corresponding to the corresponding label of present image.

Optionally, described device further include:

Return module, for not existing and meeting preset condition in judging the corresponding label of each target image When label, the mechanical arm mobile module is triggered.

Optionally, the label determining module, is specifically used for:

By target image input by the convolutional neural networks training module target convolution nerve net that training is completed in advance Network obtains the image of the target image so that the target convolutional neural networks carry out feature extraction to the target image It is corresponding to obtain each target image according to the corresponding relationship of the characteristics of image of self-contained image pattern and label for feature Label；

Optionally, the convolutional neural networks training module includes:

Model construction unit, for constructing initial convolutional neural networks；

Image pattern acquiring unit changes the fortune of the mechanical arm for the target object to be placed in predeterminated position Dynamic direction obtains the target pair of the camera in the mechanical arm according to the mobile when institute continuous collecting of each direction of motion The multiple images sample of elephant；

Label generation unit, when for according to acquiring each image pattern, the location information of the mechanical arm and the mesh The location information of mark object determines the corresponding label of each image pattern according to default label create-rule；

Model training unit, for described image sample and its corresponding label to be inputted the initial convolutional neural networks It is trained；

Training unit is completed, the value for the objective function when the initial convolutional neural networks no longer changes or the figure When the accuracy rate of decent corresponding output result reaches default accuracy rate, training is completed, is obtained described comprising described image sample The target convolutional neural networks of the corresponding relationship of characteristics of image originally and label.

Optionally, described device further include:

Second judgment module, for being moved to the corresponding position of the target mechanical arm location information in the control mechanical arm After setting, judge whether existing label is identical as default end-tag, if so, triggering handling module；

The handling module grabs the target object for controlling the mechanical arm.

Optionally, described device further include:

Successful information output module, for when the mechanical arm grabs target object success, output to be grabbed successfully Information.

A kind of control equipment, including processor, communication interface, memory and communication bus, wherein processor, communication connect Mouthful, memory completes mutual communication by communication bus；

Memory, for storing computer program；

Processor when for executing the program stored on memory, realizes any of the above-described method and step.

A kind of computer readable storage medium is stored with computer program in the computer readable storage medium, described Any of the above-described method and step is realized when computer program is executed by processor.

In scheme provided by the embodiment of the present invention, control equipment control mechanical arm is mobile default according to predetermined movement direction Distance, and it is every in the target image of the target object of the mobile when institute continuous collecting of the mechanical arm, and acquisition to obtain camera The location information for opening mechanical arm when target image carries out feature extraction to target image, and the image for obtaining target image is special Sign, according to the corresponding relationship of preset characteristics of image and label, obtains the corresponding label of each target image, then judges each In the corresponding label of target image, if there is the label for meeting preset condition, if it does, determining the existing label of acquisition The target position information of mechanical arm when corresponding target image, control mechanical arm are moved to the corresponding position of target position information. When controlling manipulator motion, control mechanical arm carries out primary mobile over long distances according to predetermined movement direction first, then herein Continuous collecting target image in moving process determines the optimal location in long range moving process herein according to target image, so Control mechanical arm returns to the optimal location afterwards, as a result, by way of two secondary control manipulator motions, mechanical arm can be made mobile To predeterminated position, it is not necessary to make manipulator motion to predeterminated position by way of repeatedly controlling, greatly reduce control mechanical arm The time of movement.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.

Fig. 1 is the first flow chart of the method for control manipulator motion provided by the embodiment of the present invention；

Fig. 2 is the flow chart that predetermined movement direction is obtained provided by the embodiment of the present invention；

Fig. 3 is the flow chart of the training method of target convolutional neural networks provided by the embodiment of the present invention；

Fig. 4 is second of flow chart of the method for control manipulator motion provided by the embodiment of the present invention；

Fig. 5 is a kind of structural schematic diagram for the device for controlling manipulator motion provided by the embodiment of the present invention；

Fig. 6 is a kind of structural schematic diagram for controlling equipment provided by the embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.

In order to reduce the time of control mechanical arm operation, the embodiment of the invention provides a kind of sides for controlling manipulator motion Method, device, control equipment and computer readable storage medium.

First below to it is provided in an embodiment of the present invention it is a kind of control manipulator motion method be introduced.

Firstly the need of explanation, provided by the embodiment of the present invention it is a kind of control manipulator motion method can apply In any control equipment for establishing communication connection with mechanical arm, it is to be understood that can be sent out between control equipment and mechanical arm Send data and instruction.The control equipment can be the electronic equipments such as computer, be not specifically limited herein.

It controls equipment also to communicate to connect with camera, under normal circumstances, which is installed on mechanical arm, for acquiring The image of target object, certainly, the camera can also be installed on any position that can collect target object image, This is not specifically limited.

As shown in Figure 1, a kind of method for controlling manipulator motion, applied to the control equipment communicated to connect with mechanical arm, Equipment is controlled also to communicate to connect with camera, this method comprises:

S101: control mechanical arm is according to the mobile pre-determined distance in predetermined movement direction, and it is mobile in mechanical arm to obtain camera When institute's continuous collecting target object target image, and acquisition every target image when mechanical arm location information.

It is understood that camera is for shooting target object, according to the actual situation, which can be with For mechanical arm object to be crawled, or the mechanical arm object or person to be tracked, this is all reasonable.

In order to grab target object or tracking target object, it may be predetermined that the direction where target object, by this Direction is as predetermined movement direction.In order to reduce control manipulator motion time, control equipment need to control mechanical arm according to Move pre-determined distance in the predetermined movement direction.

Wherein, which can determine according to the moving scene of actual machine arm, for example, it is assumed that target object It is placed on workbench, so that mechanical arm is grabbed.If mechanical arm tail end stand distance objective object is farther out, that Pre-determined distance can be set larger, such as can be 20 centimetres, 25 centimetres, 30 centimetres etc..If mechanical arm tail end is parked Positional distance target object is closer, then pre-determined distance can be set it is smaller, such as can be 10 centimetres, 8 centimetres, 5 centimetres Deng being not specifically limited herein.

For example, it is assumed that predetermined movement direction is that the right, pre-determined distance is 10 centimetres, then control equipment can be controlled Mechanical arm processed moves right 10 centimetres.

Since target object is located at predetermined movement direction, so available camera continues when mechanical arm is mobile The target image of the target object of acquisition.

Wherein, there are many modes of camera continuous collecting target image, as long as can achieve continuous uninterrupted sampling Mode all can be used as the mode of continuous collecting target image.It illustratively, can be 1 second acquisition 15-30 frame, or according to Mechanical arm movement speed determines one frame of mobile how many distance acquisitions, such as: assuming that the movement speed of mechanical arm is 15cm/s, then take the photograph As the mode of head continuous collecting target image can acquire 1 frame for mobile 1cm.

It is understood that obtain camera acquisition target image when, mechanical arm current location be it is known, because This, is when mechanical arm is mobile, the location information of mechanical arm when every target image of available acquisition.

For example, in mechanical arm moving process, the data of acquisition can indicate control equipment are as follows: (I_1, S_1), (I_2, S_2) ... ..., (I_n, S_n) }, wherein I_n represents n-th Target Photo, and S_n, which is represented, opens Target Photo in acquisition n-th When, the status information of mechanical arm itself such as location information and posture information, due to the mechanical arm in mechanical arm moving process Posture is held essentially constant, therefore, in this application, the location information of S_n main representative mechanical arm.

S102: feature extraction is carried out to target image, the characteristics of image of target image is obtained, according to preset characteristics of image With the corresponding relationship of label, the corresponding label of each target image is obtained, wherein label is for identifying mechanical arm and target object Relative positional relationship.

The corresponding label of each target image in order to obtain obtains a large amount of image pattern, wherein image pattern in advance The image of the target object as obtained in advance, target object is located at a variety of different positions, target object in image pattern Form in image pattern can not also be identical.For example, target object is a cup, in image pattern, which can Can be in positions such as image pattern center, edges, cup may be upright, handstands, traverse, the oblique forms such as put in image pattern.

It is understood that mechanical arm current location and target object position are when obtaining image pattern Know, in this way, control equipment can be according to mechanical arm current location and target object position, that is, mechanical arm and mesh The relative positional relationship for marking object, determines the corresponding label of each image pattern, in this way, label then can be used for identifying mechanical arm With the relative positional relationship of target object.

Since the characteristics of image of each image pattern is different, after the corresponding label of each image pattern has been determined, It is assured that the characteristics of image of image pattern and the corresponding relationship of label, the image for pre-setting image pattern as a result, is special The corresponding relationship of sign and label.

After controlling the above-mentioned target image of equipment acquisition, feature extraction can be carried out to target image, obtain target image Characteristics of image obtains mesh then according to the characteristics of image of the corresponding relationship and target image of preset characteristics of image and label The corresponding label of logo image.

It is subsequent to carry out citing introduction to the specific generating mode of label in order to which scheme understands and is laid out clearly.

S103: judge in the corresponding label of each target image, if there is the label for meeting preset condition, if so, Step S104 is executed, if not, without any processing.

After control equipment obtains the corresponding label of each target image, need to judge the corresponding label of each target image In, if there is the label for meeting preset condition.

Under normal circumstances, the relative positional relationship of which is identified mechanical arm and target object Can be with are as follows: mechanical arm tail end at a distance from target object within a preset range.If target object is that mechanical arm is to be crawled Object then can carry out grasping movement to object at this time；If target object is the mechanical arm object or person to be tracked, this When, mechanical arm can show deliberate action to avoid being found.Therefore, moving to acquisition, this meets preset condition to mechanical arm It is optimal state at position when the corresponding image of label, optimum state can be above-mentioned most suitable crawl object, or, most suitable Deliberate action etc. is shown in conjunction.The preset range can be determined according to factors such as the sizes of target object, be not specifically limited herein.

If there is the label for meeting preset condition, then control equipment can execute step S104 at this time.

S104: the target position information of mechanical arm when determining the existing label of acquisition corresponding target image.

Control equipment illustrates that mechanical arm is moving to the acquisition label pair when determining the label for existing and meeting preset condition It is optimal state at position when the image answered, such as: the optimal state, which can be, is most suitable for crawl object, or, most suitable Deliberate action etc. is shown in conjunction, it is thus necessary to determine that the corresponding optimal location of the optimum state.

The location information of mechanical arm due to when obtaining target image, while when obtaining acquisition target image, therefore, It can determine the target position information of mechanical arm when acquiring the corresponding target image of the existing label.

S105: control mechanical arm is moved to the corresponding position of target position information.

After target position information has been determined, control equipment can control mechanical arm and be moved to target position information correspondence Position, that is, optimal location.To carry out subsequent step, such as: crawl object shows deliberate action etc..

It is understood that execute a step S101- step S105, mechanical arm just carried out a long distance movement with An and return movement.That is control equipment control manipulator motion to the corresponding position of target position information controls in total Manipulator motion twice, compared to the mode for needing repeatedly control manipulator motion, greatly reduces control manipulator motion Time.

As it can be seen that control equipment control mechanical arm is moved according to predetermined movement direction in scheme provided by the embodiment of the present invention Dynamic pre-determined distance, and camera is obtained in the target image of the target object of the mobile when institute continuous collecting of the mechanical arm, and The location information for acquiring mechanical arm when every target image carries out feature extraction to target image, obtains target image Characteristics of image obtains the corresponding label of each target image, then sentences according to the corresponding relationship of preset characteristics of image and label Break in the corresponding label of each target image, if there is the label for meeting preset condition, if it does, determining present in acquisition Label corresponding target image when mechanical arm target position information, it is corresponding that control mechanical arm is moved to target position information Position.When controlling manipulator motion, control mechanical arm carries out primary mobile over long distances according to predetermined movement direction first, then The continuous collecting target image in this moving process determines the optimal position in long range moving process herein according to target image It sets, then controls mechanical arm and return to the optimal location, as a result, by way of two secondary control manipulator motions, machinery can be made Arm is moved to predeterminated position, it is not necessary to make manipulator motion to predeterminated position by way of repeatedly controlling, greatly reduce control The time of manipulator motion.

It describes in detail below by a specific embodiment to method shown in Fig. 1:

Such as: assuming that predetermined movement direction is that the left, presetting moving distance is 10cm, it is assumed that the every movement 1cm of mechanical arm is adopted Collect an image；

Control equipment control mechanical arm is moved to the left 10cm, and obtains camera in the mobile when institute continuous collecting of mechanical arm The target image of target object, and acquisition every target image when mechanical arm location information, obtain { (I_1, S_1), (I_ 2, S_2) ... ..., (I_10, S_10) }, the 1st Target Photo that camera acquires when wherein I_1 represents manipulator motion 1cm, S_ 1 represents the location information of mechanical arm when acquiring 1 Target Photo；

Feature extraction is carried out to target image (I_1-I_10), obtains the characteristics of image of target image (I_1-I_10), root According to the corresponding relationship of preset characteristics of image and label, the corresponding label of each target image is obtained；

Assuming that the corresponding target image of the label for meeting preset condition is the 7th target image, the 7th target image of acquisition is determined When mechanical arm target position information；

Control mechanical arm is moved to the corresponding position of target position information, i.e., on the left of mechanical arm initial position at 7cm.

Control equipment control mechanical arm is moved to the left 10cm as a result, then the 3cm that moves right is returned on the left of initial position At 7cm.Control equipment controls manipulator motion twice altogether.

Referring to fig. 2, in an implementation of the embodiment of the present invention, the process for obtaining predetermined movement direction, can wrap It includes:

S201: the present image at the target object current time of camera acquisition is obtained.

It is understood that the present image is sent after the present image at camera acquisition target object current time To control equipment, control equipment also just obtains the present image.It should be noted that target object referred to can be machine Tool arm object to be crawled, or the mechanical arm object or person to be tracked are not only limited in this certainly, do not do herein Any restriction.

In one embodiment, camera can acquire the image of target object in real time, and acquired image is sent out It send to control equipment, then it is present image that control equipment, which receives the image at target object current time,.Another real It applies in mode, camera can acquire the image at target object current time when receiving the shooting instruction of control equipment, and It is sent to control equipment, then, control equipment also can receive the image, i.e. present image.Certainly, camera can also be real When acquire the image of target object, when receiving the acquisition instruction of control equipment, then by current time target pair collected The present image of elephant is sent to control equipment, and this is also rational.

S202: feature extraction is carried out to present image, the characteristics of image of present image is obtained, according to preset characteristics of image With the corresponding relationship of label, the corresponding label of present image is obtained.

The corresponding label of present image in order to obtain obtains a large amount of image pattern in advance, wherein image pattern is The image of the target object obtained in advance, target object is located at a variety of different positions in image pattern, and target object is being schemed Form in decent can not also be identical.For example, target object is a cup, in image pattern, which may be The positions such as image pattern center, edge, cup may be upright, handstands, traverse, the oblique forms such as put in image pattern.

After controlling the above-mentioned present image of equipment acquisition, feature extraction can be carried out to present image, obtain present image Characteristics of image is worked as then according to the characteristics of image of the corresponding relationship and present image of preset characteristics of image and label The corresponding label of preceding image.

It is above-mentioned that feature extraction is carried out to present image, the characteristics of image of present image is obtained, then according to preset image The characteristics of image of the corresponding relationship and present image of feature and label, obtain the corresponding label of present image mode have it is more Kind, it is including but not limited to following several:

First way: configuration mode identification

Identification is carried out to present image and achievees the purpose that feature extraction, the characteristics of image of present image is obtained, then passes through Preset matching degree calculation assesses the characteristics of image of present image and the matching degree of each mode, wherein one mode A kind of corresponding relationship including characteristics of image and label, to obtain the corresponding label of present image according to the mode being matched to.

The second way: convolutional neural networks

The present image is inputted into the target convolutional neural networks that training is completed in advance, so that target convolutional neural networks pair Present image carries out feature extraction, the characteristics of image of present image is obtained, according to the characteristics of image of self-contained image pattern The corresponding label of present image is obtained in turn with the corresponding relationship of label.

Specifically, target convolutional neural networks are as follows: it is based on image pattern and its corresponding label, it is first to what is constructed in advance Beginning convolutional neural networks are trained obtained convolutional neural networks.Wherein, the target pair that image pattern as obtains in advance The image of elephant, target object is located at a variety of different positions, form of the target object in image pattern in image pattern It can not be identical.For example, target object is a cup, in image pattern, which may be in image pattern center, edge Equal positions, cup may be upright, handstands, traverse, the oblique forms such as put in image pattern.

Label is then used to identify the relative positional relationship of mechanical arm and target object.It is understood that obtaining image When sample, mechanical arm current location and target object position are known, in this way, control equipment can be according to machinery Arm current location and target object position, that is, the relative positional relationship of mechanical arm and target object, determine each figure Decent corresponding label, in this way, label then can be used for identifying the relative positional relationship of mechanical arm and target object.

In this way, including the characteristics of image of image pattern and pair of label by the target convolutional neural networks that training obtains It should be related to, in turn, present image is inputted target convolutional neural networks by control equipment, and target convolutional neural networks can basis It includes the characteristics of image of image pattern and the corresponding relationship of label and present image characteristics of image, currently schemed As corresponding label.

It is subsequent the specific training method of target convolutional neural networks to be lifted in order to which scheme understands and is laid out clearly Example is introduced.

S203: it according to the corresponding relationship of preset label and the direction of motion, determines corresponding to the corresponding label of present image Predetermined movement direction.

In order to determine that the direction of motion of mechanical arm, control equipment, can bases after determining the corresponding label of present image The corresponding relationship of preset label and the direction of motion determines predetermined movement direction corresponding to the corresponding label of present image.

Due to the relative positional relationship of tag identifier mechanical arm and target object, then control equipment can pre-establish The corresponding relationship of label and the direction of motion, the corresponding direction of motion of each label.As an example it is assumed that label is number 0- 26, totally 27, label 1 identifies the relative positional relationship of mechanical arm and target object are as follows: target object is located at mechanical arm tail end 30 degree of directions under left avertence, then label 1 and the corresponding relationship of the direction of motion are are as follows: 30 degree of directions under the corresponding left avertence of label 1.Into And when label is 1, target direction of motion is 30 degree of directions under left avertence.

The corresponding label of present image is obtained by way of feature extraction as a result, then according to preset label and fortune The corresponding relationship in dynamic direction, determines predetermined movement direction.

As a kind of embodiment of the embodiment of the present invention, step S103 judgement, which does not exist, in Fig. 1 meets preset condition Label when, return to step S101.

It is understood that do not meet the label of preset condition if do not existed, illustrates not exist mechanical arm and adopt moving to It is optimal state at position when collecting the corresponding image of a certain label, at this point, to control the optimal shape that equipment finds mechanical arm The corresponding optimal location of state can both continue to control at this time mechanical arm according to the mobile pre-determined distance in predetermined movement direction, and repetition is held The process of row control manipulator motion；It can also reaffirm and obtain the preset direction of motion, is i.e. execution S201-S203.

As a result, in control mechanical arm in a long distance movement, the optimum state for not finding mechanical arm is corresponding optimal When position, continues to control a mechanical arm long distance movement of progress, until finding the corresponding optimal location of optimum state, and control Mechanical arm is moved to found optimal location.

It is above-mentioned that feature extraction is carried out to target image, the characteristics of image of target image is obtained, according to preset characteristics of image With the corresponding relationship of label, obtain there are many modes of the corresponding label of each target image, including but not limited to following several:

First way: configuration mode identification

Identification is carried out to target image and achievees the purpose that feature extraction, the characteristics of image of target image is obtained, then passes through Preset matching degree calculation assesses the characteristics of image of target image and the matching degree of each mode, wherein one mode A kind of corresponding relationship including characteristics of image and label, to obtain the corresponding label of target image according to the mode being matched to.

The second way: convolutional neural networks

Target image is inputted into the target convolutional neural networks that training is completed in advance, so that target convolutional neural networks are to mesh Logo image carry out feature extraction, obtain the characteristics of image of target image, according to the characteristics of image of self-contained image pattern with The corresponding relationship of label obtains the corresponding label of each target image in turn.

In this way, including the characteristics of image of image pattern and pair of label by the target convolutional neural networks that training obtains It should be related to, in turn, target image is inputted target convolutional neural networks by control equipment, and target convolutional neural networks can basis It includes the characteristics of image of image pattern and the corresponding relationship of label and target image characteristics of image, obtain target figure As corresponding label.

As a kind of embodiment of the embodiment of the present invention, as shown in figure 3, the training side of above-mentioned target convolutional neural networks Formula may comprise steps of:

S301: initial convolutional neural networks are constructed.

It is understood that then control equipment instructs it firstly the need of one initial convolutional neural networks of building Practice, and then obtains target convolutional neural networks.In one embodiment, it includes more for can use caffe tools build one The initial convolutional neural networks of a convolutional layer.

S302: being placed in predeterminated position for target object, changes the direction of motion of mechanical arm, obtains camera in mechanical arm According to the multiple images sample of the target object of the mobile when institute continuous collecting of each direction of motion.

Image pattern is the image of the target object of camera acquisition, under normal circumstances, the target in each image pattern Object is located at a variety of different positions, and form of the target object in image pattern can also be different.In this way, image pattern can be with The feature of target object in a variety of forms is characterized, the initial convolutional neural networks of subsequent training are convenient for.For example, target object is one A cup, in image pattern, which may be in positions such as image pattern center, edges, and cup may in image pattern For upright, handstand, traverse, tiltedly put etc. forms.The conditions such as light when acquiring image pattern are also possible to different.

When acquiring above-mentioned multiple images sample, target object can be placed in predeterminated position, then change mechanical arm The direction of motion, such as: the direction of motion be forward, to the left and to the right.To make to install camera harvester on the robotic arm Tool arm moves the image pattern of the target object of when institute continuous collecting according to each direction of motion.For example, can be by target object It is placed on the platform of station etc, then controls mechanical arm and constantly change the direction of motion, and then each side can be obtained Upward multiple images sample.

Since the image pattern obtained in the unit time is more, the target convolutional neural networks trained are more accurate, and this Apply when obtaining image pattern, is the image pattern of continuous collecting target object in each direction of motion, compared to Target object is placed on each different location, then acquires the mode of image pattern again, the application obtains within the unit time Image pattern it is more, therefore, the target convolutional neural networks trained by the application are more accurate.

S303: when according to acquiring each image pattern, the location information of mechanical arm and the location information of target object, according to Default label create-rule, determines the corresponding label of each image pattern.

It is understood that mechanical arm current location and target object position are equal when obtaining each image pattern It is known, in this way, control equipment can be according to mechanical arm current location and target object position, according to default label Create-rule determines the corresponding label of each image pattern.

Specifically, in one embodiment, the location information of target object can be expressed as (x1, y1, z1), mechanical The location information of arm can be expressed as (x2, y2, z2), determine that the mode of the corresponding label of each image pattern can be with are as follows:

When according to acquiring each image pattern, x1 in the location information of x2 and target object in the location information of mechanical arm The size relation of size relation, the size relation of y2 and y1 and z2 and z1 determines each figure according to default label create-rule Decent corresponding label.

In general, (x1, y1, z1) and (x2, y2, z2) can be respectively target object center and machine in environment coordinate system The coordinate of tool arm end.The environment coordinate system can be preset three-dimensional system of coordinate, as long as target object and machinery can be indicated The position of arm, is not specifically limited herein.

That is, be directed to each image pattern, control equipment can according to the coordinate of current time mechanical arm tail end and The size relation of three coordinate values in the coordinate at target object center, to determine the corresponding label of image pattern.It is understood that , since the coordinate of mechanical arm tail end and the coordinate at target object center characterize mechanical arm and target object position, Therefore the label generated according to the two size relation is the relative positional relationship for identifying mechanical arm and target object.

S304: image pattern and its corresponding label are inputted into initial convolutional neural networks and are trained.

After the corresponding label of each image pattern has been determined, control equipment can be by image pattern and its corresponding label Above-mentioned initial convolutional neural networks are inputted to be trained.Specifically, initial convolutional neural networks are according to the image of image pattern Feature predicts its corresponding label, clear in order to describe, in this step by initial convolutional neural networks according to image pattern The label of characteristics of image prediction is known as prediction label, and the corresponding label of image pattern determined in above-mentioned steps S303 is referred to as true Real label.

After initial convolutional neural networks obtain the prediction label of image pattern, by its true tag with the image pattern into Row comparison is calculated the difference value of the two by objective function predetermined, and passes through back propagation tune according to the difference value The parameter of whole initial convolutional neural networks.In the training process, all image patterns can be looped through, and constantly adjustment is initial The parameter of convolutional neural networks.

It can be using any backpropagation mode in the related technology, herein not for the specific implementation of back propagation It is specifically limited and illustrates.The expression of mode and objective function to objective function, can be according to crawl precision Etc. factors set, be not specifically limited herein.

S305: when the value of the objective function of initial convolutional neural networks no longer changes or the corresponding output result of image pattern Accuracy rate when reaching default accuracy rate, complete training, obtain the corresponding relationship of the characteristics of image comprising image pattern and label Target convolutional neural networks.

When the value of the objective function of initial convolutional neural networks no longer changes or the corresponding output result of image pattern When accuracy rate reaches default accuracy rate, illustrate that initial convolutional neural networks at this time can be adapted for most of image sample This, obtain accurately as a result, so can deconditioning, no longer adjust the parameter of initial convolutional neural networks, and then obtain Target convolutional neural networks, it is to be understood that the target convolutional neural networks that training obtains include that the image of image pattern is special The corresponding relationship of sign and label.

Wherein, default accuracy rate can be determined according to the accuracy needed for crawl, for example, can for 85%, 90%, 95% etc., it is not specifically limited herein.

As it can be seen that being trained by above-mentioned training method to initial convolutional neural networks, available includes image pattern Characteristics of image and label corresponding relationship target convolutional neural networks, pass through the available figure of target convolutional neural networks As corresponding label, and then determine the direction of motion of mechanical arm.

Referring to fig. 4, in Fig. 1 after step S105, a kind of side controlling manipulator motion provided in an embodiment of the present invention Method can also include:

S106: judging whether existing label is identical as default end-tag, if so, executing step S107.

After control mechanical arm is moved to the corresponding position of target position information, illustrates that mechanical arm has returned to and transport over long distances Optimum state corresponding optimal location when dynamic is directed to for crawl object, which is not necessarily suitable for grabbing, and therefore, is The optimal location is determined if appropriate for crawl, control equipment need to judge existing label whether with default end-tag phase Together.

Under normal circumstances, the relative positional relationship of which is identified mechanical arm and target object are as follows: Mechanical arm tail end is being suitble in crawl range at a distance from target object.Mechanical arm can carry out grasping movement to object at this time. This is suitble to crawl range that can be determined according to factors such as the sizes of target object, is not specifically limited herein.

If existing label is identical as default end-tag, illustrate to be appropriate for grabbing at this time, then controlling at this time Equipment can execute step S107.

S107: control mechanical arm grabs target object.

If existing label is identical as default end-tag, illustrate that it is dynamic can to carry out crawl for mechanical arm at this time Make, so control equipment can control mechanical arm crawl target object at this time, and then completes the crawl to target object.

As a result, in existing label situation identical with default end-tag, control mechanical arm grabs target object.

If existing label is different from default end-tag, illustrate to be not suitable for being grabbed at this time, mechanical arm It also needs to continue to move just be grabbed, therefore, on the basis of method shown in Fig. 4, be tied in the existing label of judgement with default When beam label difference, the above method can also include:

The present image for obtaining the target object current time of camera acquisition carries out feature extraction to present image, obtains The corresponding mark of present image is obtained according to the corresponding relationship of preset characteristics of image and label to the characteristics of image of present image Label, according to the corresponding relationship of preset label and the direction of motion, determine predetermined movement corresponding to the corresponding label of present image Direction returns to step S101.

Since existing label is different from default end-tag, illustrate to be not suitable for being grabbed at this time, mechanical arm also needs Just continuing movement can be grabbed, therefore, continue to move to control mechanical arm, it is thus necessary to determine where target object to be captured Position, so that it is determined that the direction of target object, grabs to control mechanical arm to target object.

Therefore, control equipment needs to obtain the present image at the target object current time of camera acquisition, to current figure As carrying out feature extraction, the characteristics of image of present image is obtained, according to the corresponding relationship of preset characteristics of image and label, is obtained The corresponding label of present image determines the corresponding label of present image according to the corresponding relationship of preset label and the direction of motion Corresponding predetermined movement direction determines the step S201-S203 that the process in predetermined movement direction is referred in Fig. 2, herein It repeats no more.

After determining predetermined movement direction, the i.e. controllable mechanical arm of control equipment according to predetermined movement direction it is mobile it is default away from From to execute subsequent control manipulator motion to the corresponding position of target position information, circulation executes step S101- step S105, until there is the label for meeting preset condition.It is understood that every circulation executes a step S101- step S105, Mechanical arm has just carried out a long distance movement and a return movement.

The seized condition that target object is checked in order to facilitate user, as a kind of embodiment of the embodiment of the present invention, When mechanical arm grabs target object success, the above method can also include:

Output crawl successful information.

When mechanical arm successfully grabs target object, control equipment can export crawl successful information, to prompt user It grabs successfully.Certainly, control equipment also can recorde crawl successful information, so that subsequent statistical grabs accuracy rate and success rate etc. Information.

For the concrete mode of control equipment output crawl successful information, the embodiment of the present invention is not specifically limited herein, As long as family, which can be used, can know the crawl successful information.For example, control equipment can be by showing that screen shows this Successful information is grabbed, crawl successful information can also be exported by forms such as voice broadcasts, this is all reasonable.

The specific generating mode of label is introduced below:

Be expressed as (x1, y1, z1) for the location information of target object, the location information of mechanical arm be expressed as (x2, y2, Z2 for) the case where, as a kind of embodiment of the embodiment of the present invention, default label create-rule includes:

When the location information of the location information of the mechanical arm and the target object meets default combination condition, generate The default corresponding label of combination condition, wherein the default combination condition is either condition, second in first group of preset condition In group preset condition in either condition and third group preset condition either condition combination, first group of preset condition include: | X2-x1 | no more than preset value, | x2-x1 | be greater than preset value and x2 > x1 and | x2-x1 | be greater than tri- kinds of items of preset value and x2 < x1 Part, second group of preset condition include: | y2-y1 | no more than preset value, | y2-y1 | be greater than preset value and y2 > y1 and | y2- Y1 | be greater than tri- kinds of conditions of preset value and y2 < y1, the third group preset condition includes: | z2-z1 | no more than preset value, | z2- Z1 | be greater than preset value and z2 > z1 and | z2-z1 | be greater than tri- kinds of conditions of preset value and z2 < z1.

Specifically, the coordinate at target object center is (x1, y1, z1), and the coordinate of mechanical arm tail end is (x2, y2, z2), So | x2-x1 | at a distance from illustrating target object and mechanical arm tail end in the direction of the x axis.Similarly, | y2-y1 | illustrate mesh Mark object and mechanical arm tail end in the y-axis direction at a distance from, | z2-z1 | illustrate target object and mechanical arm tail end in z-axis side Upward distance.

So, as | x2-x1 | when being not more than preset value, illustrate target object and mechanical arm tail end in the direction of the x axis away from It is close, as | x2-x1 | when being greater than preset value, illustrate target object and mechanical arm tail end in the direction of the x axis at a distance from farther out, that At this point, if x2 > x1, illustrates in the direction of the x axis, target object is on the right side of mechanical arm tail end, if x2 < x1, illustrates in x In axis direction, target object is on the left of mechanical arm tail end.

Likewise, working as | y2-y1 | when being not more than preset value, illustrate target object and mechanical arm tail end in the y-axis direction Apart from close, as | y2-y1 | when being greater than preset value, illustrate target object and mechanical arm tail end in the y-axis direction at a distance from farther out, So at this point, if y2 > y1, illustrates in the y-axis direction, target object is in front of mechanical arm tail end, if y2 < y1, explanation In the y-axis direction, target object is at mechanical arm tail end rear.When | z2-z1 | when being not more than preset value, illustrate target object and machine The distance of tool arm in the z-axis direction is close, when | z2-z1 | when being greater than preset value, illustrate target object and mechanical arm tail end in z-axis Distance on direction farther out, then at this point, illustrating that in the z-axis direction, target object is on mechanical arm tail end if z2 > z1 Side, if z2 < z1, illustrates that in the z-axis direction, target object is below mechanical arm tail end.

It should be noted that above-mentioned preset value can basis if target object is mechanical arm object to be crawled It grabs the factors such as type, the size of precision and target object and determines that, if target object is smaller, which can It is smaller, for example, 3 centimetres, 5 centimetres, 7 centimetres etc.；If target object is larger, which can be larger, for example, It 10 centimetres, 15 centimetres, 18 centimetres etc., is not specifically limited herein.Certain preset value may be set to be 0, then | x2-x1 | When no more than preset value, i.e., | x2-x1 | it is 0, illustrates at this time in the direction of the x axis, in the position and target object of mechanical arm tail end The position of the heart is overlapped, and it is higher to grab precision at this time.

If target object is the mechanical arm object or person to be tracked, above-mentioned preset value can be according to tracking tightness And the factors such as type, size of target object determine, wherein mechanical arm and tracked object when tracking tightness refers to tracking Distance, high apart from small tracking tightness, the big tracking tightness of distance is low.If target object is object, since object will not It was found that being tracked, high tracking tightness can be taken at this time, then the preset value can be arranged smaller, for example, 3 centimetres, 5 Centimetre, 7 centimetres etc.；If target object is behaved, in order to avoid being found, then the preset value can be arranged larger, such as 1 Rice, 1.5 meters, 2 meters etc., are not specifically limited herein.

So it is understood that presetting item for above-mentioned first group of preset condition, second group of preset condition and third group Part includes three conditions in every group of preset condition, can be combined into 27 kinds of default combination conditions in this way.27 kinds of preset groups Conjunction condition corresponds to 27 kinds of positional relationships of mechanical arm and target object, and 27 kinds of positional relationships are by above-mentioned mechanical arm tail end and target The coordinate value of object centers determines.The corresponding 27 kinds of labels of this 27 kinds default combination conditions, in one embodiment, this 27 kinds marks Label can be number 0-26, and certainly, which can also use the label of other forms, as long as above-mentioned 27 kinds can be indicated Positional relationship, for example, can be a1, a2 ... a27 etc., this is all reasonable.

For example, by taking target object is mechanical arm object to be crawled as an example, if certain default combination condition includes: | x2- X1 | no more than preset value, | y2-y1 | no more than preset value and | z2-z1 | be greater than tri- kinds of conditions of preset value and z2 < z1, then saying Bright target object at this time is located at the lower section of mechanical arm tail end in the z-axis direction, if the default corresponding label of combination condition is 5, then it is understood that the corresponding direction of motion of label 5 is underface.In another example certain default combination condition includes: | X2-x1 | no more than preset value, | y2-y1 | no more than preset value and | z2-z1 | be not more than preset value, then illustrating target at this time Object is close at a distance from mechanical arm tail end, can carry out grasping movement, if the default corresponding label of combination condition is 0, So it is understood that label 0 is above-mentioned default end-tag.

As it can be seen that the label generated by above-mentioned label create-rule, can identify 27 kinds of positions of mechanical arm and target object Relationship is set, corresponding 27 directions of motion of 27 labels can be with according to this 27 labels during controlling manipulator motion Obtain the optimal movement direction of current time mechanical arm.

Corresponding to above method embodiment, the embodiment of the invention also provides a kind of devices for controlling mechanical movement.

The device for being provided for the embodiments of the invention a kind of control manipulator motion below is introduced.

As shown in figure 5, a kind of device for controlling manipulator motion, sets applied to the control communicated to connect with the mechanical arm Standby, the control equipment is also communicated to connect with camera, the apparatus may include:

Mechanical arm mobile module 401 for controlling the mechanical arm according to the mobile pre-determined distance in predetermined movement direction, and obtains Take the camera in the target image of the target object of the mobile when institute continuous collecting of the mechanical arm, and every target of acquisition The location information of mechanical arm when image；

Label determining module 402 obtains the image of the target image for carrying out feature extraction to the target image Feature obtains the corresponding label of each target image, wherein described according to the corresponding relationship of preset characteristics of image and label Label is used to identify the relative positional relationship of the mechanical arm Yu the target object；

First judgment module 403, for judging in the corresponding label of each target image, if it is default to there is satisfaction The label of condition, if so, triggering target position information determining module 404；

The target position information determining module 404, when for determining the corresponding target image of the existing label of acquisition The target position information of the mechanical arm；

Control module 405 is moved to the corresponding position of the target position information for controlling the mechanical arm.

As a kind of embodiment of the embodiment of the present invention, described device can also include obtaining module, the acquisition mould For obtaining the predetermined movement direction, the acquisition module may include: block

As a kind of embodiment of the embodiment of the present invention, described device can also include:

As a kind of embodiment of the embodiment of the present invention, the label determining module 402 can be specifically used for:

As a kind of embodiment of the embodiment of the present invention, the convolutional neural networks training module may include:

The handling module grabs the target object for controlling the mechanical arm.

The embodiment of the invention also provides a kind of control equipment, as shown in fig. 6, include processor 601, communication interface 602, Memory 603 and communication bus 604, wherein processor 601, communication interface 602, memory 603 are complete by communication bus 604 At mutual communication,

Memory 603, for storing computer program；

Processor 601 when for executing the program stored on memory 603, realizes following steps:

The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.

Communication interface is for the communication between above-mentioned electronic equipment and other equipment.

Memory may include random access memory (Random Access Memory, RAM), also may include non-easy The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also To be storage device that at least one is located remotely from aforementioned processor.

Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete Door or transistor logic, discrete hardware components.

As a kind of embodiment of the embodiment of the present invention, the process in the predetermined movement direction is obtained, may include:

As a kind of embodiment of the embodiment of the present invention, the method can also include:

It is described that feature extraction is carried out to the target image as a kind of embodiment of the embodiment of the present invention, obtain institute It is corresponding to obtain each target image according to the corresponding relationship of preset characteristics of image and label for the characteristics of image for stating target image Label the step of, may include:

As a kind of embodiment of the embodiment of the present invention, the training method of the target convolutional neural networks be can wrap It includes:

Construct initial convolutional neural networks；

As a kind of embodiment of the embodiment of the present invention, it is mechanical that the target is moved in the control mechanical arm After the step of arm location information corresponding position, the method can also include:

Whether the existing label of judgement is identical as default end-tag；

If so, controlling the mechanical arm grabs the target object.

As a kind of embodiment of the embodiment of the present invention, when the mechanical arm grabs target object success, institute The method of stating can also include:

Output crawl successful information.

The embodiment of the invention also provides a kind of computer readable storage medium, the computer readable storage medium memory Computer program is contained, the computer program performs the steps of when being executed by processor

As it can be seen that when computer program is executed by processor, control mechanical arm is pressed in scheme provided by the embodiment of the present invention According to the mobile pre-determined distance in predetermined movement direction, and camera is obtained in the target object of the mobile when institute continuous collecting of the mechanical arm Target image, and when every target image of acquisition the mechanical arm location information, feature extraction is carried out to target image, The characteristics of image of target image is obtained, according to the corresponding relationship of preset characteristics of image and label, obtains each target image pair Then the label answered judges in the corresponding label of each target image, if there is the label for meeting preset condition, if deposited The target position information of mechanical arm, control mechanical arm are moved to when determining the existing label of acquisition corresponding target image The corresponding position of target position information.When controlling manipulator motion, control mechanical arm is carried out according to predetermined movement direction first Primary mobile over long distances, then the continuous collecting target image in this moving process, determines herein over long distances according to target image Then optimal location in moving process controls mechanical arm and returns to the optimal location, passes through two secondary control manipulator motions as a result, Mode, mechanical arm can be made to be moved to predeterminated position, it is not necessary to make manipulator motion to default position by way of repeatedly controlling It sets, greatly reduces the time of control manipulator motion.

Construct initial convolutional neural networks；

Whether the existing label of judgement is identical as default end-tag；

If so, controlling the mechanical arm grabs the target object.

Output crawl successful information.

It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.

Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method Part explanation.

The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention It is interior.

Claims

1. a kind of method for controlling manipulator motion, which is characterized in that set applied to the control communicated to connect with the mechanical arm Standby, the control equipment is also communicated to connect with camera, which comprises

The mechanical arm is controlled according to the mobile pre-determined distance in predetermined movement direction, and obtains the camera and is moved in the mechanical arm The target image of the target object of dynamic when institute continuous collecting, and the position of the mechanical arm is believed when every target image of acquisition Breath；

Feature extraction is carried out to the target image, the characteristics of image of the target image is obtained, according to preset characteristics of image With the corresponding relationship of label, obtain the corresponding label of each target image, wherein the label for identify the mechanical arm with The relative positional relationship of the target object；

If there is the label for meeting preset condition, mechanical arm when the existing label of acquisition corresponding target image is determined Target position information；

2. the method according to claim 1, wherein obtaining the process in the predetermined movement direction, comprising:

Feature extraction is carried out to the present image, the characteristics of image of the present image is obtained, according to preset characteristics of image With the corresponding relationship of label, the corresponding label of the present image is obtained；

According to the corresponding relationship of preset label and the direction of motion, determines and preset corresponding to the corresponding label of the present image The direction of motion.

3. the method according to claim 1, wherein the method also includes:

If not there is no the label for meeting preset condition, returns and execute the control mechanical arm according to the shifting of predetermined movement direction The step of dynamic pre-determined distance.

4. being obtained the method according to claim 1, wherein described carry out feature extraction to the target image The characteristics of image of the target image obtains each target image pair according to the corresponding relationship of preset characteristics of image and label The step of label answered, comprising:

The target image is inputted into the target convolutional neural networks that training is completed in advance, so that the target convolutional neural networks Feature extraction is carried out to the target image, the characteristics of image of the target image is obtained, according to self-contained image pattern Characteristics of image and label corresponding relationship, obtain the corresponding label of each target image；

Wherein, the target convolutional neural networks are as follows: image pattern and its corresponding label are based on, to the initial volume constructed in advance Product neural network is trained obtained convolutional neural networks, and the target convolutional neural networks include described image sample The corresponding relationship of characteristics of image and label.

5. according to the method described in claim 4, it is characterized in that, the training method of the target convolutional neural networks, comprising:

Construct initial convolutional neural networks；

The target object is placed in predeterminated position, changes the direction of motion of the mechanical arm, obtains the camera in institute Mechanical arm is stated according to the multiple images sample of the target object of the mobile when institute continuous collecting of each direction of motion；

When according to acquiring each image pattern, the location information of the location information of the mechanical arm and the target object, according to Default label create-rule, determines the corresponding label of each image pattern；

When the value of the objective function of the initial convolutional neural networks no longer changes or the corresponding output result of described image sample Accuracy rate when reaching default accuracy rate, complete training, obtain the characteristics of image comprising described image sample and label The target convolutional neural networks of corresponding relationship.

6. the method according to claim 1, wherein being moved to the target machine in the control mechanical arm After the step of tool arm location information corresponding position, the method also includes:

Whether the existing label of judgement is identical as default end-tag；

If so, controlling the mechanical arm grabs the target object.

7. according to the method described in claim 6, it is characterized in that, when the mechanical arm grab the target object success when, The method also includes:

Output crawl successful information.

8. a kind of device for controlling manipulator motion, which is characterized in that set applied to the control communicated to connect with the mechanical arm Standby, the control equipment is also communicated to connect with camera, and described device includes:

Mechanical arm mobile module, for controlling the mechanical arm according to the mobile pre-determined distance in predetermined movement direction, and described in acquisition Camera moves the target image of the target object of when institute continuous collecting in the mechanical arm, and when every target image of acquisition The location information of the mechanical arm；

Label determining module obtains the characteristics of image of the target image, root for carrying out feature extraction to the target image According to the corresponding relationship of preset characteristics of image and label, the corresponding label of each target image is obtained, wherein the label is used for Identify the relative positional relationship of the mechanical arm Yu the target object；

First judgment module, for judging in the corresponding label of each target image, if exist and meet preset condition Label, if so, triggering target position information determining module；

The target position information determining module, machinery when for determining the existing label of acquisition corresponding target image The target position information of arm；

9. a kind of control equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing Device, communication interface, memory complete mutual communication by communication bus；

Memory, for storing computer program；

Processor when for executing the program stored on memory, realizes method and step as claimed in claim 1 to 7.

10. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Program, the computer program realize method and step as claimed in claim 1 to 7 when being executed by processor.