CN106393144B - The method and system that vision tracks under a kind of multirobot operation mode - Google Patents

The method and system that vision tracks under a kind of multirobot operation mode Download PDF

Info

Publication number
CN106393144B
CN106393144B CN201611056367.0A CN201611056367A CN106393144B CN 106393144 B CN106393144 B CN 106393144B CN 201611056367 A CN201611056367 A CN 201611056367A CN 106393144 B CN106393144 B CN 106393144B
Authority
CN
China
Prior art keywords
target
vision
delta
under
robot
Prior art date
Application number
CN201611056367.0A
Other languages
Chinese (zh)
Other versions
CN106393144A (en
Inventor
李丽丽
王亚梅
曹永军
余学文
龙建佑
魏杰
Original Assignee
顺德职业技术学院
华南智能机器人创新研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 顺德职业技术学院, 华南智能机器人创新研究院 filed Critical 顺德职业技术学院
Priority to CN201611056367.0A priority Critical patent/CN106393144B/en
Publication of CN106393144A publication Critical patent/CN106393144A/en
Application granted granted Critical
Publication of CN106393144B publication Critical patent/CN106393144B/en

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00Controls for manipulators
    • B25J13/08Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls

Abstract

The invention discloses the method and system that vision under a kind of multirobot operation mode tracks, its robot control system includes kinetic control system, vision system, the kinetic control system is connect with vision system, and the kinetic control system is connected at least Liang Tai DELTA robots, is connected by EtherCAT buses between the kinetic control system and at least Liang Tai DELTA robots.Through the embodiment of the present invention, by establishing the video tracking under co-ordination pattern, view-based access control model system realizes video tracking image of the target object under multiple DELTA robots under cooperative work mode, and just there is distance measuring method that can predict in time target object, the follow-up next state value of target object in time, it ensure that the orderly monitoring of whole operation Object Process, it can be according to sequential course replay whole operation process.

Description

The method and system that vision tracks under a kind of multirobot operation mode

Technical field

The present invention relates to intelligent Manufacturing Technology fields, and in particular to the side that vision tracks under a kind of multirobot operation mode Method and system.

Background technology

With the continuous development of robot technology, more and more robots start to substitute the various tasks of mankind's execution.Machine Device people is to automatically control machine(Robot)Be commonly called as, it includes all simulation human behaviors or thought and simulation to automatically control machine The machinery of other biological(Such as robot dog, Doraemon etc.).There are many classification and dispute to the definition of robot in the narrow sense, has A little computer programs or even also referred to as robot.In contemporary industry, robot refers to the man-made machine dress that can execute task automatically It sets, to replace or assist human work.Highly emulated robot in ideal is advanced integral traffic control opinion, mechano-electronic, calculating Machine and artificial intelligence, materialogy and bionic product, scientific circles research and develop to this direction at present, but robot is remote Process control is still not perfect, and the application of big data is universal not yet, and also in off-line state, robot is deep for the data acquisition of robot Degree study also comes from the storage of native data.

DELTA robots are one kind of parallel robot, and DELTA robots drive 3 parallel four sides by outer revolute pair Shape branch, along with an intermediate rotation drive shaft, it can be achieved that the space four-dimensional movement of its end effector, configures vision system Afterwards, the extensive use in the sorting packaging of electronics, light industry, food and medicine and other fields.

The control system of current existing each producer DELTA robots is to use the special purpose robot's controller of its own It is this bad using the control system system versatility of nonshared control unit come the robot control system built.General DELTA Robot control system is made of kinetic control system, servo-drive system, Visual Tracking System and conveyer belt tracking system, is such as schemed In shown in 1, since there are multiple DELTA machines person cooperative works, target object different times under different DELTA robots Interior completion different operation process, during how realizing video tracking, and the prior art lacks a kind of feasible program.

Invention content

The present invention provides the method and system that vision under a kind of multirobot operation mode tracks, and coordinate work by foundation Video tracking under operation mode solves existing machine vision and coordinates sex chromosome mosaicism.

The present invention provides the methods that vision under a kind of multirobot operation mode tracks, wherein:Robot control system Including kinetic control system, vision system, the kinetic control system and vision system and the kinetic control system connect There are at least Liang Tai DELTA robots, passes through between the kinetic control system and at least Liang Tai DELTA robots EtherCAT buses are connected, the method includes:

During kinetic control system controls DELTA robot manipulations, vision system builds each DELTA robot Under target tracking model, target object image is acquired and is handled;

The mode of operation of each DELTA robot is detected and is identified with the method for image procossing, is obtained every Target object under one DELTA robot manipulation's process;

Using distance measuring method, it is accurately positioned the distance between DELTA robots and target object, and to target object state It is predicted;

Vision system tracks the next state value of target object in time according to prediction.

The vision system builds the target tracking model under each DELTA robot, is carried out to target object image It acquires with processing and includes:

Track algorithm based on DELTA robots working background color combining and edge establishes target tracking model.

The method with image procossing is detected and identifies packet to the mode of operation of each DELTA robot It includes:

The target object under the mode of operation to each DELTA robot is detected and is known using frame differential method Not.

It is described to utilize distance measuring method, it is accurately positioned the distance between DELTA robots and target object, and to target object State carries out prediction:

The distance between DELTA robots and target object are calculated using monocular vision distance measuring method;

In the motion state for predicting target object state based on Kalman filtering algorithm.

According to prediction, the tracking next state value of target object includes the vision system in time:

Based on the Collaborative Control instruction between at least Liang Tai DELTA robots, real-time tracing is carried out to target object, And the state value under view-based access control model system grabs difference DELTA robots.

Correspondingly, the present invention also provides a kind of robot control systems, including:Kinetic control system, vision system institute State that kinetic control system is connected with vision system and the kinetic control system is connected at least Liang Tai DELTA robots, institute It states and is connected by EtherCAT buses between kinetic control system and at least Liang Tai DELTA robots, wherein:

Kinetic control system carries out coordinated manipulation for control targe object under at least Liang Tai DELTA robots;

Vision system is used to build the target tracking model under each DELTA robot, is carried out to target object image Acquisition and processing;The mode of operation of each DELTA robot is detected and is identified with the method for image procossing, is obtained Target object under each DELTA robot manipulation's process;Using distance measuring method, it is accurately positioned DELTA robots and target The distance between object, and target object state is predicted;Vision system is according to one under prediction in time tracking target object A state value.

Track algorithm of the vision system based on DELTA robots working background color combining and edge is established target and is chased after Track model.

The vision system is using frame differential method to the target object under the mode of operation to each DELTA robot It is detected and identifies.

The vision system using monocular vision distance measuring method calculate between DELTA robots and target object away from From;In the motion state for predicting target object state based on Kalman filtering algorithm.

The vision system is based on the Collaborative Control instruction between at least Liang Tai DELTA robots, to target object Carry out real-time tracing, and the state value under view-based access control model system grabs difference DELTA robots.

In the present invention, by establishing the video tracking under co-ordination pattern, view-based access control model system realizes target object Video tracking image under multiple DELTA robots under cooperative work mode, and just have distance measuring method can be in time to target Object is predicted that follow up the next state value of target object in time, ensure that the orderly monitoring of whole operation Object Process, can With according to sequential course replay whole operation process.

Description of the drawings

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with Other attached drawings are obtained according to these attached drawings.

Fig. 1 is the structural schematic diagram of robot control system in the prior art;

Fig. 2 is the robot control system architecture schematic diagram in the embodiment of the present invention;

Fig. 3 is the method flow diagram that vision tracks under the multirobot operation mode in the embodiment of the present invention.

Specific implementation mode

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those of ordinary skill in the art are obtained all other without creative efforts Embodiment shall fall within the protection scope of the present invention.

Specifically, Fig. 2 shows the robot control system architecture schematic diagram in the embodiment of the present invention, robot control System includes kinetic control system, vision system, which is connect with vision system and the motion control system System is connected at least Liang Tai DELTA robots, the kinetic control system and this at least pass through between Liang Tai DELTA robots EtherCAT buses are connected, and three DELTA robots are shown in the embodiment of the present invention, wherein:

Kinetic control system carries out coordinated manipulation for control targe object under at least Liang Tai DELTA robots;

Vision system is used to build the target tracking model under each DELTA robot, is carried out to target object image Acquisition and processing;The mode of operation of each DELTA robot is detected and is identified with the method for image procossing, is obtained Target object under each DELTA robot manipulation's process;Using distance measuring method, it is accurately positioned DELTA robots and target The distance between object, and target object state is predicted;Vision system is according to one under prediction in time tracking target object A state value.

In specific implementation process, tracking of the vision system based on DELTA robots working background color combining and edge Algorithm establishes target tracking model;And using frame differential method to the mesh under the mode of operation to each DELTA robot Mark object is detected and identifies;Using monocular vision distance measuring method calculate between DELTA robots and target object away from From;In the motion state for predicting target object state based on Kalman filtering algorithm;Based on at least Liang Tai DELTA robots Between Collaborative Control instruction, real-time tracing is carried out to target object, and under view-based access control model system grabs difference DELTA robots State value.

It should be noted that the track algorithm at color combining and edge establishes target tracking model in the embodiment of the present invention, Some Color edge detection algorithms may be used, it has fully considered the local edge of coloured image, using color difference carry out with Track can extract more colour edging information to compensate for the deficiency for losing edge when the detection of conventional edge detection method, And accuracy of detection and effect are all more satisfactory, have certain practical value and good treatment effect.

It should be noted that frame differential method be it is a kind of by adjacent two frame in sequence of video images make calculus of differences come The method for obtaining moving target profile, it can be perfectly suitable for the presence of multiple moving targets and video camera movement. When occurring abnormal object of which movement in monitoring scene, more apparent difference is will appear between frame and frame, two frames subtract each other, and obtain two Frame brightness of image absolute value of the difference, judges whether it more than threshold value analyzes the kinetic characteristic of video or image sequence, determines figure As whether there is or not object of which movement in sequence.The difference of image sequence frame by frame is equivalent to the high pass filter carried out to image sequence under time domain Wave.

It should be noted that monocular vision distance measuring method can utilize pinhole imaging system principle, imaging point and target point are obtained Mapping relations, establish national forest park in Xiaokeng.Then by the analysis to target image, the face of object and target image is obtained Product mapping relations, establish the straight line ranging model of vision measurement;By image procossing, the characteristic point of target image is extracted, by light The distance relation of the heart and object is converted into the distance relation of optical center and characteristic point, it is proposed that the monocular vision of feature based point is surveyed Away from principle.

It should be noted that Kalman filtering algorithm is a kind of utilization linear system state equation, inputted by system defeated Go out to observe data, the algorithm of optimal estimation is carried out to system mode, to realize the prediction process of data value.

Correspondingly, Fig. 3 shows the method flow that vision tracks under the multirobot operation mode in the embodiment of the present invention Figure, wherein:Robot control system includes kinetic control system, vision system, and the kinetic control system connects with vision system It connects and the kinetic control system is connected at least Liang Tai DELTA robots, the kinetic control system and described at least two It is connected by EtherCAT buses between platform DELTA robots, this method comprises the following steps:

S301, kinetic control system control DELTA robot manipulations during, vision system builds each DELTA Target tracking model under robot is acquired and handles to target object image;

In specific implementation process, built by the track algorithm based on DELTA robots working background color combining and edge Vertical target tracking model.The track algorithm at color combining and edge establishes target tracking model in the embodiment of the present invention, can adopt With some Color edge detection algorithms, it has fully considered the local edge of coloured image, using color difference into line trace, thus The deficiency for losing edge when the detection of conventional edge detection method is compensated for, more colour edging information, Er Qiejian can be extracted It surveys precision and effect is all more satisfactory, there is certain practical value and good treatment effect.

S302, the mode of operation of each DELTA robot is detected and is identified with the method for image procossing, obtained Obtain the target object under each DELTA robot manipulation's process;

In specific implementation process, using frame differential method to the target under the mode of operation to each DELTA robot Object is detected and identifies.Frame differential method is one kind by being obtained as calculus of differences to adjacent two frame in sequence of video images The method for obtaining moving target profile, it can be perfectly suitable for the presence of multiple moving targets and video camera movement.When When occurring abnormal object of which movement in monitoring scene, more apparent difference is will appear between frame and frame, two frames subtract each other, and obtain two frames Brightness of image absolute value of the difference, judges whether it more than threshold value analyzes the kinetic characteristic of video or image sequence, determines image Whether there is or not object of which movement in sequence.The difference of image sequence frame by frame is equivalent to the high-pass filtering carried out to image sequence under time domain. Kalman filtering algorithm is a kind of utilization linear system state equation, data is observed by system input and output, to system mode The algorithm for carrying out optimal estimation, to realize the prediction process of data value.

S303, using distance measuring method, be accurately positioned the distance between DELTA robots and target object, and to target pair As state is predicted;

In specific implementation process, calculated between DELTA robots and target object using monocular vision distance measuring method Distance;In the motion state for predicting target object state based on Kalman filtering algorithm.

Monocular vision distance measuring method can utilize pinhole imaging system principle, obtain the mapping relations of imaging point and target point, build Vertical national forest park in Xiaokeng.Then by the analysis to target image, the area mapping relations of object and target image is obtained, are built The straight line ranging model of vertical vision measurement;By image procossing, extract the characteristic point of target image, by optical center and object away from From the distance relation that transformation is optical center and characteristic point, it is proposed that the monocular vision range measurement principle of feature based point.

S304, vision system track the next state value of target object in time according to prediction.

In specific implementation process, based on the Collaborative Control instruction between at least Liang Tai DELTA robots, to target pair As carrying out real-time tracing, and the state value under view-based access control model system grabs difference DELTA robots.

To sum up, by establishing the video tracking under co-ordination pattern, view-based access control model system realizes target object multiple Video tracking image under DELTA robots under cooperative work mode, and just have distance measuring method can in time to target object into Row prediction, follow up the next state value of target object in time, ensure that the orderly monitoring of whole operation Object Process, can be according to Sequential course replay whole operation process.

One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is can It is completed with instructing relevant hardware by program, which can be stored in computer readable storage medium, and storage is situated between Matter may include:Read-only memory(ROM, Read Only Memory), random access memory(RAM, Random Access Memory), disk or CD etc..

The method and system that vision tracks under multirobot operation mode are provided for the embodiments of the invention above to carry out It is discussed in detail, principle and implementation of the present invention are described for specific case used herein, above example Explanation be merely used to help understand the present invention method and its core concept;Meanwhile for those of ordinary skill in the art, According to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion in this specification Appearance should not be construed as limiting the invention.

Claims (10)

1. a kind of method that vision tracks under multirobot operation mode, which is characterized in that wherein:Robot control system includes Kinetic control system, vision system, the kinetic control system is connect with vision system and kinetic control system connection There are at least Liang Tai DELTA robots, passes through between the kinetic control system and at least Liang Tai DELTA robots EtherCAT buses are connected, the method includes:
During kinetic control system controls DELTA robot manipulations, vision system is built under each DELTA robot Target tracking model is acquired and handles to target object image;
The mode of operation of each DELTA robot is detected and is identified with the method for image procossing, obtains each Target object under DELTA robot manipulation's processes;
Using distance measuring method, it is accurately positioned the distance between DELTA robots and target object, and carry out to target object state Prediction;
Vision system tracks the next state value of target object in time according to prediction.
2. the method that vision tracks under multirobot operation mode as described in claim 1, which is characterized in that the vision system Target tracking model under each DELTA robot of system structure, is acquired target object image and includes with processing:
Track algorithm based on DELTA robots working background color combining and edge establishes target tracking model.
3. the method that vision tracks under multirobot operation mode as described in claim 1, which is characterized in that described with figure The method of picture processing is detected to the mode of operation of each DELTA robot and identification includes:
The target object under the mode of operation of each DELTA robot is detected and is identified using frame differential method.
4. the method that vision tracks under multirobot operation mode as described in claim 1, which is characterized in that described to utilize survey Away from method, it is accurately positioned the distance between DELTA robots and target object, and prediction is carried out to target object state and includes:
The distance between DELTA robots and target object are calculated using monocular vision distance measuring method;
In the motion state for predicting target object based on Kalman filtering algorithm.
5. such as the method that vision tracks under Claims 1-4 any one of them multirobot operation mode, which is characterized in that According to prediction, the tracking next state value of target object includes the vision system in time:
Based on the Collaborative Control instruction between at least Liang Tai DELTA robots, real-time tracing, and base are carried out to target object The state value of the target object under different DELTA robots is captured in vision system.
6. a kind of robot control system, which is characterized in that including:Kinetic control system, vision system, the motion control system System is connected with vision system and the kinetic control system is connected at least Liang Tai DELTA robots, the motion control system System is connected between at least Liang Tai DELTA robots by EtherCAT buses, wherein:
Kinetic control system carries out coordinated manipulation for control targe object under at least Liang Tai DELTA robots;
Vision system is used to build the target tracking model under each DELTA robot, is acquired to target object image With processing;The mode of operation of each DELTA robot is detected and is identified with the method for image procossing, is obtained each Target object under platform DELTA robot manipulation's processes;Using distance measuring method, it is accurately positioned DELTA robots and target object The distance between, and target object state is predicted;Vision system tracks the next shape of target object in time according to prediction State value.
7. robot control system as claimed in claim 6, which is characterized in that the vision system is based on DELTA robots The track algorithm at working background color combining and edge establishes target tracking model.
8. robot control system as claimed in claim 6, which is characterized in that the vision system utilizes frame differential method pair Target object under the mode of operation of each DELTA robot is detected and identifies.
9. robot control system as claimed in claim 6, which is characterized in that the vision system uses monocular vision ranging Method calculates the distance between DELTA robots and target object;Target object is being predicted based on Kalman filtering algorithm Motion state.
10. such as claim 6 to 9 any one of them robot control system, which is characterized in that the vision system is based on institute The Collaborative Control instruction between at least Liang Tai DELTA robots is stated, real-time tracing, and view-based access control model system are carried out to target object Capture the state value of the target object under different DELTA robots.
CN201611056367.0A 2016-11-26 2016-11-26 The method and system that vision tracks under a kind of multirobot operation mode CN106393144B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611056367.0A CN106393144B (en) 2016-11-26 2016-11-26 The method and system that vision tracks under a kind of multirobot operation mode

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611056367.0A CN106393144B (en) 2016-11-26 2016-11-26 The method and system that vision tracks under a kind of multirobot operation mode

Publications (2)

Publication Number Publication Date
CN106393144A CN106393144A (en) 2017-02-15
CN106393144B true CN106393144B (en) 2018-09-04

Family

ID=58082673

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611056367.0A CN106393144B (en) 2016-11-26 2016-11-26 The method and system that vision tracks under a kind of multirobot operation mode

Country Status (1)

Country Link
CN (1) CN106393144B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108827250A (en) * 2018-05-07 2018-11-16 深圳市三宝创新智能有限公司 A kind of robot monocular vision ranging technology method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345258A (en) * 2013-06-16 2013-10-09 西安科技大学 Target tracking method and system of football robot
CN103406905A (en) * 2013-08-20 2013-11-27 西北工业大学 Robot system with visual servo and detection functions
CN104589357A (en) * 2014-12-01 2015-05-06 佛山市万世德机器人技术有限公司 Control system and method of DELTA robots based on visual tracking
CN204819543U (en) * 2015-06-24 2015-12-02 燕山大学 Centralized control formula multirobot motion control system
CN106097322A (en) * 2016-06-03 2016-11-09 江苏大学 A kind of vision system calibration method based on neutral net

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5815761B2 (en) * 2014-01-23 2015-11-17 ファナック株式会社 Visual sensor data creation system and detection simulation system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103345258A (en) * 2013-06-16 2013-10-09 西安科技大学 Target tracking method and system of football robot
CN103406905A (en) * 2013-08-20 2013-11-27 西北工业大学 Robot system with visual servo and detection functions
CN104589357A (en) * 2014-12-01 2015-05-06 佛山市万世德机器人技术有限公司 Control system and method of DELTA robots based on visual tracking
CN204819543U (en) * 2015-06-24 2015-12-02 燕山大学 Centralized control formula multirobot motion control system
CN106097322A (en) * 2016-06-03 2016-11-09 江苏大学 A kind of vision system calibration method based on neutral net

Also Published As

Publication number Publication date
CN106393144A (en) 2017-02-15

Similar Documents

Publication Publication Date Title
Mehta et al. Vision-based control of robotic manipulator for citrus harvesting
Zhang et al. Towards vision-based deep reinforcement learning for robotic motion control
US9452531B2 (en) Controlling a robot in the presence of a moving object
Ishii et al. 2000 fps real-time vision system with high-frame-rate video recording
Reina et al. Ambient awareness for agricultural robotic vehicles
Weiss et al. Plant detection and mapping for agricultural robots using a 3D LIDAR sensor
JP4333364B2 (en) Robot device, face recognition method, and face recognition device
Luo et al. Vision-based extraction of spatial information in grape clusters for harvesting robots
Kim et al. SLAM-driven robotic mapping and registration of 3D point clouds
CN102842148B (en) Method and device for capturing markerless motion and reconstructing scene
CN108594851A (en) A kind of autonomous obstacle detection system of unmanned plane based on binocular vision, method and unmanned plane
Linder et al. On multi-modal people tracking from mobile platforms in very crowded and dynamic environments
JP6514166B2 (en) Machine learning apparatus, robot system and machine learning method for learning robot operation program
US20120271173A1 (en) Automatic ultrasonic scanning system and scanning method thereof
Tully et al. Leap-frog path design for multi-robot cooperative localization
KR20120025582A (en) Method of creating teaching data for robot, and teaching system for robot
CN104589356A (en) Dexterous hand teleoperation control method based on Kinect human hand motion capturing
Tölgyessy et al. The Kinect sensor in robotics education
CN103170973A (en) Man-machine cooperation device and method based on Kinect video camera
CN103049912B (en) Random trihedron-based radar-camera system external parameter calibration method
CN103271784B (en) Man-machine interactive manipulator control system and method based on binocular vision
CN102914967A (en) Autonomous navigation and man-machine coordination picking operating system of picking robot
DE102015002760A1 (en) Robot simulation system that simulates the process of removing workpieces
Mustafah et al. Stereo vision images processing for real-time object distance and size measurements
WO2017167282A1 (en) Target tracking method, electronic device, and computer storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant