CN109493369A - A kind of intelligent robot vision dynamic positioning tracking and system - Google Patents
A kind of intelligent robot vision dynamic positioning tracking and system Download PDFInfo
- Publication number
- CN109493369A CN109493369A CN201811058413.XA CN201811058413A CN109493369A CN 109493369 A CN109493369 A CN 109493369A CN 201811058413 A CN201811058413 A CN 201811058413A CN 109493369 A CN109493369 A CN 109493369A
- Authority
- CN
- China
- Prior art keywords
- target
- product
- image
- video
- carried out
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004519 manufacturing process Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 8
- 238000003384 imaging method Methods 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 239000003814 drug Substances 0.000 abstract description 3
- 229940079593 drug Drugs 0.000 abstract description 3
- 238000001914 filtration Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010054949 Metaplasia Diseases 0.000 description 1
- 210000000481 breast Anatomy 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000015689 metaplastic ossification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Manipulator (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of intelligent robot vision dynamic positioning trackings, the described method comprises the following steps: S10, feature information extraction step；S20, video acquisition step；S30, video-processing steps, S40, object identification step；S50, fining identification step, S60, target locating step.Utilize the present invention, it may be implemented in the sorting that multi-assortment production in intelligent production process carries out Classification and Identification and substandard product, production efficiency can be effectively improved and reduce production cost, produce and be of great significance particularly with the intelligence of the industries such as food, drug, flexibility.
Description
Technical field
The present invention relates to a kind of positioning and tracing method more particularly to a kind of intelligent robot vision dynamic positioning trackings
And system.
Background technique
Computer vision is how a research makes the science of machine " seeing " further just refer to and use video camera
It replaces human eye the machine vision such as to be identified, tracked and measured to target with computer, and further does graphics process, make at computer
Reason becomes the image for being more suitable for eye-observation or sending instrument detection to.Mainly it is divided to two classes currently based on the localization method of vision,
Two-dimensional visual positioning is positioned with three-dimensional modeling.Two-dimensional visual localization method be with monocular to control to as and surrounding enviroment carry out
Then vision calibration carries out precise manipulation by the coordinate demarcated.And three-dimensional modeling positioning is mainly taken the photograph using 2 or more
As head shoots target, shot image is merged, to establish the three-dimensional coordinate of target, emulates three-dimensional environment, it is real
Now to the accurate operation of target.
Industrial robot is widely used in industrial production, under the guide operation of staff, can complete many
Instruction, but robot does not perceive the ability of extraneous information, is unable to adjust the working environment having changed, leads to industrial production
Things quality and precision are by serious influence.Therefore, the Import computer vision technique in industrial machine production, can be improved
The precision of industrial robot operation and the function of realizing real-time tracking correction, can satisfy the reality in production process to robot
The requirement of when property, and industrial robot is made preferably to adapt to complicated site environment.
Currently, domestic many enterprises can realize intelligent production, but in the Classification and Identification of product, unqualified production
Product are selected etc. there is also many deficiencies, so that the intelligent production of product is by many limitations.
Summary of the invention
In order to overcome the deficiencies of the prior art and there are the problem of, the present invention provides a kind of intelligent robot vision dynamic positioning
Tracking and system may make point that product is realized in intelligent production process using technical solution provided by the invention
Class identification is selected with substandard product.
The present invention is achieved by the following technical solutions: a kind of intelligent robot vision dynamic positioning tracking, institute
State method the following steps are included:
S10, feature information extraction step, classify to product, and carry out the form of multi-faceted, more scenes to different
Product carries out Image Acquisition, carries out the block diagram of target object and removes dryness processing, therefrom extract the characteristic information of target product, and builds
Vertical training set；
S20, video acquisition step obtain the production video flowing and/or figure of product in production process using imaging sensor
Piece；
S30, video-processing steps pre-process the production video flowing and/or picture, to generate convenient for processing
Image is serialized, while further including calibration training set processing；
S40, object identification step are handled the image of serializing according to the product model pre-established, and to figure
Product as in is demarcated；
Calibration treated product is carried out classification processing, and similar target is carried out essence by S50, fining identification step
Refinement identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize
Target following；
S60, target locating step, to by fining identification step, treated that image positions in video
Tracking.
Preferably, in the target locating step, according to previous frame image come future position, while according to working as
Preceding image detects target position；Then using the metric form of fusion, according to target predicted position and detection position in geneva
The COS distance of performance characteristic between distance and borderline region in space, then cascade matching is carried out using Hungary Algorithm,
To finally realize locating and tracking.
Preferably, using Kalman filtering come future position, and utilizing target in the target locating step
Detection algorithm detects target position.
According to the inventive concept of the above method, the present invention also provides a kind of tracking of intelligent robot vision dynamic positioning to be
System, the system comprises have:
Characteristic information extracting module for classifying to product, and carries out the form of multi-faceted, more scenes to different
Product carries out Image Acquisition, carries out the block diagram of target object and removes dryness processing, therefrom extract the characteristic information of target product, and builds
Vertical training set；
Video acquiring module, for obtaining the production video flowing and/or figure of product in production process using imaging sensor
Piece；
Video processing module, for being pre-processed to the production video flowing and/or picture, to generate convenient for processing
Image is serialized, while being also used to demarcate training set processing；
Object identification module, for handling according to the product model pre-established the image of serializing, and to figure
Product as in is demarcated；
Identification module is refined, product will carry out classification processing for that will demarcate that treated, and similar target be carried out smart
Refinement identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize
Target following；
Target locating module, for treated that image positions in video by fining identification step
Tracking.
Preferably, in the target locating module, according to previous frame image come future position, while according to working as
Preceding image detects target position；Then using the metric form of fusion, according to target predicted position and detection position in geneva
The COS distance of performance characteristic between distance and borderline region in space, then cascade matching is carried out using Hungary Algorithm,
To finally realize locating and tracking.
Preferably, using Kalman filtering come future position, and utilizing target in the target locating module
Detection algorithm detects target position.
Using the present invention, it can be achieved that multi-assortment production carries out Classification and Identification and substandard product in intelligent production process
Sorting, can effectively improve production efficiency and reduce production cost, particularly with the intelligent, flexible of the industries such as food, drug
Metaplasia production is of great significance.
Detailed description of the invention
Fig. 1 is the flow diagram of the method for the embodiment of the present invention.
Fig. 2 is the modular structure schematic diagram of system described in the embodiment of the present invention.
Fig. 3 is the simplified process signal that target locating module described in the embodiment of the present invention realizes target locating
Figure.
Specific embodiment
For the ease of the understanding of those skilled in the art, the present invention is made below in conjunction with the drawings and specific embodiments further
Detailed description.
As shown in Fig. 1, a kind of intelligent robot vision dynamic positioning tracking, the described method comprises the following steps:
S10, feature information extraction step are classified to product and (manual sort can be used), and carry out multi-faceted, more
The form of scape carries out Image Acquisition to different products, carries out the block diagram of target object and removes dryness processing, therefrom extracts target and produces
The characteristic information of product, and establish training set；In the present embodiment, the characteristic information includes the shape information and appearance of target product
The characteristic informations such as profile information；
S20, video acquisition step obtain the production video flowing and/or figure of product in production process using imaging sensor
Piece；
S30, video-processing steps pre-process the production video flowing and/or picture, to generate convenient for processing
Image is serialized, while further including calibration training set processing；
S40, object identification step are handled the image of serializing according to the product model pre-established, and to figure
Product as in is demarcated；
Calibration treated product is carried out classification processing, and similar target is carried out essence by S50, fining identification step
Refinement identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize
Target following；
S60, target locating step, to by fining identification step, treated that image positions in video
Tracking.
As preferred embodiment, in the target locating step, according to previous frame image come future position,
Target position is detected according to present image simultaneously；Then using the metric form of fusion, according to target predicted position and detection
Position in the distance in mahalanobis space and the COS distance of performance characteristic between borderline region, then using Hungary Algorithm into
Row cascade matching, to finally realize locating and tracking.In the present embodiment, in the target locating step, Kalman is utilized
Filtering carrys out future position, and detects target position using algorithm of target detection.
According to the inventive concept of the above method, the embodiment of the invention also provides a kind of intelligent robot vision dynamic positioning
Tracking system, the system are as shown in Fig. 2 comprising have:
Characteristic information extracting module for classifying to product, and carries out the form of multi-faceted, more scenes to different
Product carries out Image Acquisition, carries out the block diagram of target object and removes dryness processing, therefrom extract the characteristic information of target product, and builds
Vertical training set；The characteristic information includes the characteristic informations such as shape information and the appearance profile information of target product；
Video acquiring module, for obtaining the production video flowing and/or figure of product in production process using imaging sensor
Piece；
Video processing module, for being pre-processed to the production video flowing and/or picture, to generate convenient for processing
Image is serialized, while being also used to demarcate training set processing；
Object identification module, for handling according to the product model pre-established the image of serializing, and to figure
Product as in is demarcated；
Identification module is refined, product will carry out classification processing for that will demarcate that treated, and similar target be carried out smart
Refinement identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize
Target following；
Target locating module, for treated that image positions in video by fining identification step
Tracking.
In a wherein preferred embodiment, the target locating module is predicted according to previous frame image
Target position, while target position is detected according to present image；Then using the metric form of fusion, according to target prediction position
Position is set and detected in the distance in mahalanobis space and the COS distance of performance characteristic between borderline region, then uses breast tooth
Sharp algorithm carries out cascade matching, to finally realize locating and tracking.In the present embodiment, in the target locating module, benefit
With Kalman filtering come future position, and target position is detected using algorithm of target detection.Target in the present embodiment
The simplified process of the realization target locating of locating and tracking module is as shown in Fig. 3.
As preferred embodiment, the video acquiring module include with video acquisition function CCD camera or
Imaging sensor with similar functions, while the video acquiring module and video processing module have data transmission interface,
So that can carry out data transmission between the video acquiring module and video processing module.The video acquiring module can pass through
It is wired or wirelessly obtain from intelligent robot peripheral apparatus CCD camera multi items object on intelligent production line
It is dynamically taken pictures or is imaged, and be transferred in video processing module by ICP/IP protocol and stored and handled, depending on
Frequency processing module pre-processes real-time video, can be believed by effective video after one section of initialization of interception or image
Breath generates clear effective serializing image by decoding, while demarcating to training set.
Using technical solution provided by the invention, it can be achieved that the product of multi items carries out classification knowledge in intelligent production process
Other and substandard product sorting can effectively improve production efficiency and reduce production cost, particularly with the row such as food, drug
Intelligence, the flexibility production of industry are of great significance.
Above-described embodiment is preferable implementation of the invention, is not limited to the present invention, is not departing from this hair
Under the premise of bright inventive concept, it is any obviously replace it is within the scope of the present invention.
Claims (8)
1. a kind of intelligent robot vision dynamic positioning tracking, which is characterized in that the described method comprises the following steps:
S10, feature information extraction step, classify to product, and carry out the form of multi-faceted, more scenes to different products
Image Acquisition is carried out, the block diagram of target object is carried out and removes dryness processing, therefrom extracts the characteristic information of target product, and establishes instruction
Practice collection；
S20, video acquisition step obtain the production video flowing and/or picture of product in production process using imaging sensor；
S30, video-processing steps pre-process the production video flowing and/or picture, to generate the sequence convenient for processing
Change image, while further including calibration training set processing；
S40, object identification step are handled the image of serializing according to the product model pre-established, and in image
Product demarcated；
Calibration treated product is carried out classification processing, and similar target is refined by S50, fining identification step
Identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize target
Tracking；
S60, target locating step, to by fining identification step treated image positioned in video with
Track.
2. according to the method described in claim 1, it is characterized by: in the target locating step, according to previous frame figure
As carrying out future position, while target position is detected according to present image；Then using the metric form of fusion, according to mesh
It marks predicted position and detects position in the distance in mahalanobis space and the COS distance of performance characteristic between borderline region, then
Cascade matching is carried out using Hungary Algorithm, to finally realize locating and tracking.
3. according to the method described in claim 2, it is characterized by: being filtered in the target locating step using Kalman
Wave carrys out future position, and detects target position using algorithm of target detection.
4. method according to any one of claims 1 to 3, it is characterised in that: the characteristic information includes that target produces
The shape information and appearance profile information of product.
5. a kind of intelligent robot vision dynamic positioning tracking system, which is characterized in that the system comprises have:
Characteristic information extracting module for classifying to product, and carries out the form of multi-faceted, more scenes to different products
Image Acquisition is carried out, the block diagram of target object is carried out and removes dryness processing, therefrom extracts the characteristic information of target product, and establishes instruction
Practice collection；
Video acquiring module, for obtaining the production video flowing and/or picture of product in production process using imaging sensor；
Video processing module, for being pre-processed to the production video flowing and/or picture, to generate the sequence convenient for processing
Change image, while being also used to demarcate training set processing；
Object identification module, for handling according to the product model pre-established the image of serializing, and in image
Product demarcated；
Identification module is refined, for the product progress classification processing that will demarcate that treated, and similar target is refined
Identification establishes video inter-frame relation, by the way that present image and previous frame image object are carried out similarity measurement to realize target
Tracking；
Target locating module, for by fining identification step treated image positioned in video with
Track.
6. system according to claim 5, it is characterised in that: in the target locating module, according to previous frame figure
As carrying out future position, while target position is detected according to present image；Then using the metric form of fusion, according to mesh
It marks predicted position and detects position in the distance in mahalanobis space and the COS distance of performance characteristic between borderline region, then
Cascade matching is carried out using Hungary Algorithm, to finally realize locating and tracking.
7. system according to claim 6, it is characterised in that: in the target locating module, filtered using Kalman
Wave carrys out future position, and detects target position using algorithm of target detection.
8. the system according to any one of claim 5~7, it is characterised in that: the characteristic information includes that target produces
The shape information and appearance profile information of product.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811058413.XA CN109493369B (en) | 2018-09-11 | 2018-09-11 | Intelligent robot vision dynamic positioning and tracking method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811058413.XA CN109493369B (en) | 2018-09-11 | 2018-09-11 | Intelligent robot vision dynamic positioning and tracking method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109493369A true CN109493369A (en) | 2019-03-19 |
| CN109493369B CN109493369B (en) | 2020-12-29 |
Family
ID=65689594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811058413.XA Active CN109493369B (en) | 2018-09-11 | 2018-09-11 | Intelligent robot vision dynamic positioning and tracking method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109493369B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110154036A (en) * | 2019-06-24 | 2019-08-23 | 山东大学 | Indoor service robot controller design method and system under vision dynamical system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101236599A (en) * | 2007-12-29 | 2008-08-06 | 浙江工业大学 | Human face recognition detection device based on multi- video camera information integration |
| US7564455B2 (en) * | 2002-09-26 | 2009-07-21 | The United States Of America As Represented By The Secretary Of The Navy | Global visualization process for personal computer platforms (GVP+) |
| CN101576956A (en) * | 2009-05-11 | 2009-11-11 | 天津普达软件技术有限公司 | On-line character detection method based on machine vision and system thereof |
| CN107169519A (en) * | 2017-05-18 | 2017-09-15 | 重庆卓来科技有限责任公司 | A kind of industrial robot vision's system and its teaching method |
| CN107516127A (en) * | 2017-08-21 | 2017-12-26 | 山东大学 | Service robot independently obtains people and wears the method and system for taking article ownership semanteme |
| CN108171748A (en) * | 2018-01-23 | 2018-06-15 | 哈工大机器人（合肥）国际创新研究院 | A kind of visual identity of object manipulator intelligent grabbing application and localization method |
| CN108363997A (en) * | 2018-03-20 | 2018-08-03 | 南京云思创智信息科技有限公司 | It is a kind of in video to the method for real time tracking of particular person |
-
2018
- 2018-09-11 CN CN201811058413.XA patent/CN109493369B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7564455B2 (en) * | 2002-09-26 | 2009-07-21 | The United States Of America As Represented By The Secretary Of The Navy | Global visualization process for personal computer platforms (GVP+) |
| CN101236599A (en) * | 2007-12-29 | 2008-08-06 | 浙江工业大学 | Human face recognition detection device based on multi- video camera information integration |
| CN101576956A (en) * | 2009-05-11 | 2009-11-11 | 天津普达软件技术有限公司 | On-line character detection method based on machine vision and system thereof |
| CN107169519A (en) * | 2017-05-18 | 2017-09-15 | 重庆卓来科技有限责任公司 | A kind of industrial robot vision's system and its teaching method |
| CN107516127A (en) * | 2017-08-21 | 2017-12-26 | 山东大学 | Service robot independently obtains people and wears the method and system for taking article ownership semanteme |
| CN108171748A (en) * | 2018-01-23 | 2018-06-15 | 哈工大机器人（合肥）国际创新研究院 | A kind of visual identity of object manipulator intelligent grabbing application and localization method |
| CN108363997A (en) * | 2018-03-20 | 2018-08-03 | 南京云思创智信息科技有限公司 | It is a kind of in video to the method for real time tracking of particular person |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110154036A (en) * | 2019-06-24 | 2019-08-23 | 山东大学 | Indoor service robot controller design method and system under vision dynamical system |
| CN110154036B (en) * | 2019-06-24 | 2020-10-13 | 山东大学 | Design method and system of indoor service robot controller under visual dynamic system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109493369B (en) | 2020-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107688391B (en) | Gesture recognition method and device based on monocular vision | |
| CN107660039B (en) | A kind of lamp control system of identification dynamic gesture | |
| CN104156726B (en) | A kind of workpiece identification method and device based on geometric characteristic | |
| CN102982557B (en) | Method for processing space hand signal gesture command based on depth camera | |
| CN107992881A (en) | A kind of Robotic Dynamic grasping means and system | |
| CN108942923A (en) | A kind of mechanical arm crawl control method | |
| CN108536157A (en) | A kind of Intelligent Underwater Robot and its system, object mark tracking | |
| CN106022213A (en) | Human body motion recognition method based on three-dimensional bone information | |
| CN102930334B (en) | Video recognition counter for body silhouette | |
| EP3120294A1 (en) | System and method for motion capture | |
| CN102704215A (en) | Automatic cutting method of embroidery cloth based on combination of DST file parsing and machine vision | |
| CN104821010A (en) | Binocular-vision-based real-time extraction method and system for three-dimensional hand information | |
| CN104281839A (en) | Body posture identification method and device | |
| CN110110797B (en) | Water surface target training set automatic acquisition method based on multi-sensor fusion | |
| CN107145226A (en) | Eye control man-machine interactive system and method | |
| CN111027432A (en) | Gait feature-based visual following robot method | |
| CN110458025A (en) | A kind of personal identification and localization method based on binocular camera | |
| CN108582075A (en) | A kind of intelligent robot vision automation grasping system | |
| Faisal et al. | Depth estimation from video using computer vision and machine learning with hyperparameter optimization | |
| CN111399634B (en) | Method and device for recognizing gesture-guided object | |
| CN207752527U (en) | A kind of Robotic Dynamic grasping system | |
| CN109137448B (en) | Intelligent fabric pattern arranging and weft straightening system based on machine vision | |
| CN115035546A (en) | Three-dimensional human body posture detection method and device and electronic equipment | |
| CN109493369A (en) | A kind of intelligent robot vision dynamic positioning tracking and system | |
| CN109636856A (en) | Object 6 DOF degree posture information union measuring method based on HOG Fusion Features operator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210513 Address after: Room 215-6, No.10, Siwei Road, Dongli Economic and Technological Development Zone, Dongli District, Tianjin Patentee after: Tianjin tieshe Intelligent Technology Co.,Ltd. Address before: 8j, block B, Konka R & D building, 28 Keji South 12th Road, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Patentee before: SHENZHEN KONGSHI INTELLIGENCE SYSTEM Co.,Ltd. |
|
| TR01 | Transfer of patent right |