CN208092786U - A kind of the System of Sorting Components based on convolutional neural networks by depth - Google Patents

A kind of the System of Sorting Components based on convolutional neural networks by depth Download PDF

Info

Publication number
CN208092786U
CN208092786U CN201820186110.5U CN201820186110U CN208092786U CN 208092786 U CN208092786 U CN 208092786U CN 201820186110 U CN201820186110 U CN 201820186110U CN 208092786 U CN208092786 U CN 208092786U
Authority
CN
China
Prior art keywords
depth
convolutional neural
neural networks
host computer
industrial robot
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201820186110.5U
Other languages
Chinese (zh)
Inventor
陈志澜
陈绪
阮宏洋
赵宏武
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jianqiao University
Original Assignee
Shanghai Jianqiao University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jianqiao University filed Critical Shanghai Jianqiao University
Priority to CN201820186110.5U priority Critical patent/CN208092786U/en
Application granted granted Critical
Publication of CN208092786U publication Critical patent/CN208092786U/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The utility model is related to the System of Sorting Components based on a kind of convolutional neural networks by depth, including switch board, industrial robot, depth camera, pneumatic cleft hand, workbench, host computer, industrial robot is through switch board controlled motion, pneumatic cleft hand is connected to the front end of industrial robot, depth camera is located at the top of workbench, is connect with host computer signal.Depth camera carries out Image Acquisition to part, after the image of acquisition is input to host computer processing, and it moves into the depth convolutional neural networks model being trained to, the depth convolutional neural networks model carries out target identification and classification to the image of acquisition, and obtained target part position coordinates and depth coordinate are sent to the switch board of industrial robot, switch board is successfully captured and is put into designated position to target part by robot is controlled, to realize the sorting to various parts, the workload for reducing manual sort improves the working efficiency of factory.

Description

A kind of the System of Sorting Components based on convolutional neural networks by depth
Technical field
The utility model is related to part Sorting Technique fields, more particularly, to one kind based on depth convolutional neural networks Image recognition and framing the System of Sorting Components.
Background technology
Huge benefit will be brought to factory to can only sorting for engineering component, save labour, and another further aspect makes Worker avoids being engaged in these repetitive works.As country proposes made in China 2025, society requires with country establishes intelligence Energy chemical plant, the mankind are freed from the dirty and messy environment of plant.
Chinese patent CN100484645A discloses substandard products automatic sorting method and equipment on high-speed automated production line, Its method is to start 3DOF after production line substandard products sorting system receives substandard products sorting signal and link transmission device, will be with The connected mechanical arm of the device quickly navigates to the surface of substandard products, and keeps synchronized movement with product line conveyer belt, so Mechanical arm declines crawl substandard products, promotion and discharges to designated position afterwards, after substandard products release, is linked transmission device by 3DOF Mechanical arm is resetted, next round sorting crawl is waited for.Its equipment includes electrical control and mechanical part;Mechanical part include 3 from By degree linkage transmission device and it is installed in the mechanical arm being made of mechanical arm arm and mechanical arm pawl on the transmission device;Mechanical arm pawl Center be provided with video camera, electric control system links transmission device with 3DOF respectively and mechanical arm is electrically connected.But It is the control field that the fuzzy control based on neural network is used for servo motor by the patent, multi-sensor information is used to obtain It takes cooperation camera image processing to obtain object centre coordinate information, reuses the fuzzy control servo motor based on neural network, To be captured to three-dimension object.And this patent uses industrial robot, first need not to the motor of robot into Row control;Secondly above-mentioned patent uses the conventional method (geometric properties extracted based on manual features in image processing field Extraction), the method makes camera require the condition of light high, robustness shortcoming first, is done for different working environment needs Specific adjustment, does not have an applicability, and the image procossing based on depth convolutional neural networks that this patent uses, and does not need pair Feature carries out cumbersome manual features extraction (all feature extractions exist with inside convolutional neural networks);And this patent is due to making It is depth convolutional neural networks, the method that the simulation mankind obtain image information, robustness is very high with applicability, does not need It is specifically adjusted for specific working environment;This patent uses depth camera, can obtain depth information, and cooperation is deep Degree convolutional neural networks can also carry out certain crawl to irregular object;Finally, above-mentioned patent is to single every time Object is captured, if there are multiple objects in camera fields of view, above-mentioned patent will be unable to capture, and this patent can be to a visual field Interior multiple objects carry out positioning and frame takes, and can judge that the object for being easy crawl the most is captured, and then can persistently follow Ring, until all crawl finishes object in the target visual field.
Utility model content
The purpose of this utility model is exactly to provide a kind of raising efficiency to overcome the problems of the above-mentioned prior art With the System of Sorting Components based on depth convolutional neural networks of precision.
The purpose of this utility model can be achieved through the following technical solutions:
A kind of the System of Sorting Components based on convolutional neural networks by depth, including switch board, industrial robot, depth Camera, pneumatic cleft hand, workbench, host computer,
The industrial robot is through switch board controlled motion, before the pneumatic cleft hand is connected to industrial robot End, the depth camera are located at the top of workbench, are connect with the host computer signal.
The host computer uses the Faster-RCNN networks based on VGG depth convolutional neural networks models.
The host computer carries out the training of a large amount of part image samples before sorting system work.
The great amount of images of various parts each posture under same workbench is acquired by depth camera, it is tagged to build respectively The training set of vertical part data collection and verification collect.Training set needs image installation part category to classify, and verification collection is to upset The required part picked of classifying specifically is clapped under same workbench background and takes respective difference by the part of sequence respectively A few thousand sheets images of posture, and these image rotations, scaling, addition noise spot etc. are increased into data volume and establish part image data Collection.Image data set is inputted into depth convolutional neural networks, network is trained.
The depth camera obtains image, sends an image to and inputs after host computer processing into train completion Calculation processing in Faster-RCNN network models, to obtain the position coordinates and depth coordinate of part, by position coordinates and depth Coordinate is sent to the switch board of industrial robot, and control industrial robot reaches correct position and depth, then controls pneumatic pawl Hand captures part and is put into designated position.
The mechanical arm that the industrial machine can artificially move on three-dimensional.
Compared with prior art, the utility model effectively identifies target object using depth convolutional neural networks With detection, it is effectively reduced weight, the deviation quantity of traditional neural network by the convolution operation of network, pondization operation, carries The high convenience of model foundation.In addition, compared with the extraction identification of the morphological feature of traditional industry camera, the identification of this system Effect has more robustness and adaptability, influence that can be to avoid environment to system.
Description of the drawings
Fig. 1 is the structural schematic diagram of the utility model;
Fig. 2 is data set Establishing process figure;
Fig. 3 is the work flow diagram of the utility model.
In figure, 1- switch boards, 2- mechanical arms, 3- depth cameras, the pneumatic cleft hands of 4-, 5- parts, 6- workbench, 7- are upper Machine.
Specific implementation mode
The utility model is described in detail with reference to specific embodiment.Following embodiment will be helpful to this field Technical staff further understands the utility model, but does not limit the utility model in any form.It should be pointed out that ability For the those of ordinary skill in domain, without departing from the concept of the premise utility, various modifications and improvements can be made. These belong to the scope of protection of the utility model.
Embodiment
The part of image recognition and framing based on a kind of convolutional neural networks by depth of the utility model sorts System hardware is made of industrial robot and its switch board 1, depth camera 3, pneumatic cleft hand 4, workbench 6, host computer 7.This reality It applies in example, industrial robot is using the mechanical arm 2 that can be moved on three-dimensional, through 1 controlled motion of switch board, pneumatic cleft hand 4 It is connected to the front end of mechanical arm 2, depth camera 3 is located at the top of workbench 6, is connect with 7 signal of host computer.
Host computer 7 uses the Faster-RCNN networks based on VGG depth convolutional neural networks models, is in sorting The training of a large amount of part image samples is carried out before system work, it is specific as follows:
The utility model establishes data set firstly the need of the preparation of progress, and flow as shown in Fig. 2, use first The part 5 that classification picks required for depth camera 3 acquires is clapped under 6 background of same workbench respectively takes the several of respective different postures Thousand sheets image.Then by the image of acquisition according to the part classification of respective type, and it is tagged.Again by these image rotations, Scaling, addition noise spot etc. are to increase data volume.By the unified size such as all picture formats such as pixel quantity, part image is established Data set, in case input network.
It is further to introduce Faster-RCNN (faster range searching convolutional neural networks) network model, it will make by oneself Part data concentration training collection input this network and be trained.And simultaneously use data set in verification set pair network effect into Row verification, is finally completed the preparation of network model.
Specific above-mentioned Faster-RCNN network models training operation is divided into following steps:
1) input test image.
2) whole image is inputted into CNN (depth convolutional neural networks), carries out feature extraction.
3) it is generated with RPN and suggests window (proposals), every image generates 300 suggestion windows.
4) suggestion window is mapped on last layer of convolution feature map of CNN models.
5) each RoI is made to generate fixed-size feature map by pooling layers of RoI.
6) utilize Softmax Loss (detection class probability) and Smooth L1Loss (detection frame returns) general to classifying Rate and frame return (Bounding box regression) joint training.The depth convolutional neural networks wherein used are works The VGG network models being most widely used in journey.
Utility model works process is as shown in figure 3, include the following steps:
Depth camera 3 is demarcated first, to establish the transformational relation of work coordinate system and industrial robot coordinate system, Then depth camera 3 is taken pictures, and photo is sent to host computer 7, and subsequently host computer carries out processing to photo and complies with net Network input requirements.
Further, treated the photo of host computer 7 is incoming is completed trained network, and network will outline all identify Target, and most probable crawl target is judged, if there is no target objects on image, terminate, otherwise above-mentioned object judgement As a result OpenCV will be used to carry out the acquisition of position coordinates and depth coordinate.
Further, coordinate will be sent to mechanical arm 2 and its switch board 1 with depth information, and subsequently control machinery arm 2 arrives Up to designated position, controls pneumatic cleft hand 4 and target object is captured
Further, judge whether to capture successfully, if not successfully, mechanical arm 2 returns to origin, again return to [0026] and start to hold Row.If capturing successfully, target object is put into designated position, then mechanical arm 2 returns to origin, again returns to the bat of depth camera 3 Start to execute according to step.
If sensor judgement security risk occurs or presses stop button, the movement of mechanical arm 2 stops, and then presses beginning Button, mechanical arm 2 return to origin, and the same step of taking pictures of depth camera 3 that returns starts to execute.
Specific embodiment of the utility model is described above.It is to be appreciated that the utility model not office It is limited to above-mentioned particular implementation, those skilled in the art can make various deformations or amendments within the scope of the claims, This has no effect on the substantive content of the utility model.

Claims (4)

1. the System of Sorting Components based on a kind of convolutional neural networks by depth, which is characterized in that the system includes switch board (1), industrial robot (2), depth camera (3), pneumatic cleft hand (4), workbench (6), host computer (7),
The industrial robot (2) is connected to industrial robot through switch board (1) controlled motion, the pneumatic cleft hand (4) (2) front end, the depth camera (3) are located at the top of workbench (6), are connect with described host computer (7) signal.
2. the System of Sorting Components based on a kind of convolutional neural networks by depth according to claim 1, feature exist In the host computer (7) is the upper of Faster-RCNN network of the use based on VGG depth convolutional neural networks models Position machine.
3. the System of Sorting Components based on a kind of convolutional neural networks by depth according to claim 2, feature exist In the host computer (7) is the host computer for the training for carrying out a large amount of part image samples before sorting system work.
4. the System of Sorting Components based on a kind of convolutional neural networks by depth according to claim 1, feature exist In the industrial robot (2) is the mechanical arm that can be moved on three-dimensional.
CN201820186110.5U 2018-02-02 2018-02-02 A kind of the System of Sorting Components based on convolutional neural networks by depth Active CN208092786U (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201820186110.5U CN208092786U (en) 2018-02-02 2018-02-02 A kind of the System of Sorting Components based on convolutional neural networks by depth

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201820186110.5U CN208092786U (en) 2018-02-02 2018-02-02 A kind of the System of Sorting Components based on convolutional neural networks by depth

Publications (1)

Publication Number Publication Date
CN208092786U true CN208092786U (en) 2018-11-13

Family

ID=64063373

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201820186110.5U Active CN208092786U (en) 2018-02-02 2018-02-02 A kind of the System of Sorting Components based on convolutional neural networks by depth

Country Status (1)

Country Link
CN (1) CN208092786U (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110171708A (en) * 2019-05-10 2019-08-27 惠州市德赛电池有限公司 A kind of automation control method of high-precision feeding and blowing
CN110220275A (en) * 2019-05-06 2019-09-10 珠海格力电器股份有限公司 Control the method and the apparatus of air conditioning of apparatus of air conditioning output temperature
CN110497419A (en) * 2019-07-15 2019-11-26 广州大学 Building castoff sorting machine people
CN111906781A (en) * 2020-07-08 2020-11-10 西安交通大学 Robot autonomous tool construction method and system based on graph neural network and related equipment

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110220275A (en) * 2019-05-06 2019-09-10 珠海格力电器股份有限公司 Control the method and the apparatus of air conditioning of apparatus of air conditioning output temperature
CN110171708A (en) * 2019-05-10 2019-08-27 惠州市德赛电池有限公司 A kind of automation control method of high-precision feeding and blowing
CN110497419A (en) * 2019-07-15 2019-11-26 广州大学 Building castoff sorting machine people
CN111906781A (en) * 2020-07-08 2020-11-10 西安交通大学 Robot autonomous tool construction method and system based on graph neural network and related equipment
CN111906781B (en) * 2020-07-08 2021-07-13 西安交通大学 Robot autonomous tool construction method and system based on graph neural network and related equipment

Similar Documents

Publication Publication Date Title
CN108229665A (en) A kind of the System of Sorting Components based on the convolutional neural networks by depth
CN208092786U (en) A kind of the System of Sorting Components based on convolutional neural networks by depth
CN107392896B (en) A kind of Wood Defects Testing method and system based on deep learning
CN104992449A (en) Information identification and surface defect on-line detection method based on machine visual sense
CN207861446U (en) Control system for robot destacking apparatus
CN108280856A (en) The unknown object that network model is inputted based on mixed information captures position and orientation estimation method
CN107402221A (en) A kind of defects of display panel recognition methods and system based on machine vision
CN105666485A (en) Automatic identifying and positioning chess placing robot based on image processing
CN110314854B (en) Workpiece detecting and sorting device and method based on visual robot
CN106269548A (en) A kind of object automatic sorting method and device thereof
CN105690393A (en) Four-axle parallel robot sorting system based on machine vision and sorting method thereof
CN104483320A (en) Digitized defect detection device and detection method of industrial denitration catalyst
CN106442553A (en) Detection and recognition device and method for sprayed codes on cylindrical surfaces of copper rings
CN109726777A (en) PCB appearance detection system and detection method Internet-based
CN111266315A (en) Ore material online sorting system and method based on visual analysis
CN111421539A (en) Industrial part intelligent identification and sorting system based on computer vision
CN108748149A (en) Based on deep learning without calibration mechanical arm grasping means under a kind of complex environment
CN103148783B (en) A kind of automatic testing method of valve rocker installation site
CN205552536U (en) Four -axis parallel robot letter sorting system based on machine vision
CN111723782A (en) Deep learning-based visual robot grabbing method and system
CN111709365A (en) Automatic human motion posture detection method based on convolutional neural network
CN106645174A (en) Automatic online visual apparent-defect inspection system for general purpose engine
CN111168674A (en) Test card assembling method based on CCD positioning system
CN206416179U (en) A kind of motion target tracking positioning and grasping system based on binocular vision
Meng et al. X-DenseNet: deep learning for garbage classification based on visual images

Legal Events

Date Code Title Description
GR01 Patent grant
GR01 Patent grant