CN110385282A - Fifth wheel vision detection system and method in Automatic manual transmission based on deep learning - Google Patents
Fifth wheel vision detection system and method in Automatic manual transmission based on deep learning Download PDFInfo
- Publication number
- CN110385282A CN110385282A CN201910621434.6A CN201910621434A CN110385282A CN 110385282 A CN110385282 A CN 110385282A CN 201910621434 A CN201910621434 A CN 201910621434A CN 110385282 A CN110385282 A CN 110385282A
- Authority
- CN
- China
- Prior art keywords
- wheel
- camera
- assembly
- frame
- priori frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 230000005540 biological transmission Effects 0.000 title claims abstract description 14
- 238000012549 training Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000000284 extract Substances 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 2
- 238000005520 cutting process Methods 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 238000000605 extraction Methods 0.000 abstract 1
- 238000007781 pre-processing Methods 0.000 abstract 1
- 238000007689 inspection Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000000203 mixture Substances 0.000 description 2
- 241000237858 Gastropoda Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000002594 fluoroscopy Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 239000003446 ligand Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 239000002893 slag Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
- B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
- B07C5/3422—Sorting according to other particular properties according to optical properties, e.g. colour using video scanning devices, e.g. TV-cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/38—Collecting or arranging articles in groups
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/12—Acquisition of 3D measurements of objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses the fifth wheel vision detection systems and method in a kind of Automatic manual transmission based on deep learning.Detection system includes workbench, angle adjustable camera, light source, vision controller etc., and vision controller identifies object therein, then tell Assembly part and fifth wheel by carrying out processing analysis to the image obtained from camera；Detection method includes the following steps: to obtain multi-angle assembly area image, it is input to progress feature extraction in trained target detection network model after pre-processing to image and predicts object space and type, then judge whether each object belongs to fifth wheel, mark the position of fifth wheel and alarm.Energy indirect labor of the present invention is with multi-angle measured in real time the fifth wheel in specific region in assembling process, has the advantages that Detection accuracy is high, strong real-time, using flexible, it is possible to reduce the introducing of assembling process fifth wheel enhances product reliability.
Description
Technical field
The invention belongs to artificial intelligence fields, and in particular to the fifth wheel vision based on deep learning in a kind of Automatic manual transmission
Detection system and method.
Background technique
Fifth wheel refers to present in product by external all unrelated with product specified states entered or inside generates
Substance.In the fairly large equipment assembling process with high reliability and high security, due to device structure complexity, production
Technique and assembly process are various, are very easy to introduce fifth wheel.For example worker operation is improper may bring screw, washer, head into
The objects such as hair, rag remnants；It such as welds, machine process and may introduce the fifth wheels such as welding slag, metal fragment.These
If fifth wheel is left can leave serious security risk in a device, the normal work of highly reliable equipment may be influenced, or even is made
At failure, initiation safety accident etc..
By long-term development, current fifth wheel Detection & Controling method mainly has visual ear to listen detection method, endoscope
Detection method, x-ray fluoroscopy detection method, ultrasonic Detection Method, Ma Tela detection method and Particle Impact Noise Detection method etc., detecting step
It is more and more stringent.But most automatic testing method is only applicable to assemble complete object, and often manually examines in assembling
It looks into, that there are still human factors is larger, is easy the problems such as missing inspection.
Fifth wheel vision detection system refers to that during Automatic manual transmission, computer passes through the assembly section that detection camera obtains
Area image information identifies object therein and judges whether there is fifth wheel.Fifth wheel vision inspection process essence is one
The characteristics of target detection process, the method that mainstream algorithm of target detection is all based on deep learning at present, such method be using
Convolutional neural networks extract feature, possess strong antijamming capability, the features such as Detection accuracy is high.
Summary of the invention
In order to solve the problems in background technique, the present invention provides in a kind of assembling process based on the extra of deep learning
Object visible detection method, this method using multiple angle adjustable cameras and be equipped with light source to assembly area carry out multi-angled shooting,
Then target detection is carried out using image of the YOLOv3 network to acquisition, identifies the object in assembly area and is screened out from it
Fifth wheel.
The technical solution adopted by the invention is as follows:
One, a kind of fifth wheel vision detection system in Automatic manual transmission based on deep learning
Including workbench, angle adjustable camera, annular light source, alarm lamp, display screen and vision controller, workbench
Two sides are fixed with two vertical profiles, are connected with horizontal profile between two vertical profiles, and assembly is placed in table surface,
It is provided with annular light source right above assembly, annular light source is installed in horizontal profile by light source adjusting rod, is located at dress
Upper left side, upper right side and the surface of ligand are provided with angle adjustable camera, positioned at assembly upper left side and upper right side can
Hue angle camera is respectively arranged on two vertical profiles, and the angle adjustable camera right above assembly is installed on horizontal type
On material and it is located at right above annular light source；Vision controller, vision controller are placed in the control cabinet below workbench
It is connected respectively with the display screen being installed on above table top and the alarm lamp for being installed on bench-top.
Each angle adjustable camera includes the driving machine of camera, camera support and worm screw and turbine type camera connection sheet composition
Structure, vertical profile or horizontal profile have been bolted camera support, and the two sides positioned at the camera support back side are mounted on snail
Bar support frame is connected with worm screw between two worm screw support frames, and worm screw bottom is connected with turbine type camera connection sheet by engagement,
The helical tooth of worm screw is meshed with the gear of turbine type camera connection sheet top surface；Camera support below worm screw is provided with arc
Slot, turbine type camera connection sheet are connected by passing through the screw of arc groove with the positive camera of camera support is located at, turbine type phase
Machine connection sheet rotates under worm drive around arc groove, to drive the rotation of camera, realizes the angular adjustment of camera.
The worm screw both ends are provided with hexagonal hole, rotate the rotation that hexagonal hole realizes worm screw by allen wrench.
The vision controller shows testing result, and the control when detecting the fifth wheel in assembly by display screen
Alarm lamp alarm.
Two, the detection method of the fifth wheel vision detection system using above-mentioned based on deep learning, comprising the following steps:
S1: opening annular light source and camera, the image of assembly when three cameras are from multiple angle acquisition Automatic manual transmissions；
S2: all images that vision controller is acquired in history assembling process are pre-processed and are marked, and will mark
Image after note is divided into training set and verifying collection；
S3: carrying out clustering according to the size of training set objects in images, is arranged by clustering best first
Test frame number and best priori frame size；
S4: the training set input YOLOv3 target detection network after step S3 clustering is trained, and is being tested
It is verified on card collection, obtains the Assembly part that can identify kind of object and position and common fifth wheel identification model；
S5: vision controller by from camera real-time reception to detection image pre-process after input step S4 Assembly part
With common fifth wheel identification model, prediction obtains the type of object and position in detection image；
S6: vision controller judges whether the object in detection image belongs to current assembly according to the prediction result of step S5
The part that body uses, if not then regarding as fifth wheel, marking the position of fifth wheel and being alarmed by alarm lamp.
Pretreatment operation in the step S2 and step S5 includes carrying out cutting to image making picture traverse and height ratio
For 1:1 and carrying out linear scaling to image makes picture size be unified for 1024 × 1024.
The step S2 specifically: all images for acquiring vision controller in history assembling process are located in advance
Reason, and marks type number, centre coordinate, width and the height of each object in pretreated image, by pretreatment with
All picture construction data sets after mark, then data set is randomly divided into training set and verifying collection in 4 to 1 ratio；
The type number of the object is labeled according to the type number in object inventory, and object inventory includes vision
In controller it is all can identification objects type and its corresponding type number, the object that can be identified include Assembly part and not
Belong to the fifth wheel of Assembly part.
The step S3 specifically: extract object width and height all in all training set images of step S2 and make
For true frame, then K priori frame is set as cluster centre, priori frame number K increases one by one since 1, by all true
Real frame carries out Kmeans clustering and obtains the corresponding most short total distance D of K priori frameK, when K is bigger, DKIt is smaller, but model is pre-
The calculation amount of survey process is bigger.
Meet the priori frame number K of the following conditions as best priori frame number, illustrates that most short total distance variation is slow, this
When K priori frame as best priori frame, the corresponding priori frame size of each best priori frame is as best priori frame size:
||DK-1-DK|-|DK-DK+1| | < minimum threshold of distance
Wherein, K is priori frame number；DKTo carry out Kmeans cluster point to all true frames when priori frame number is K
Analyse obtained most short total distance；
The described process for carrying out Kmeans clusterings to all true frames is as follows: will each true frame with it is true from this
The nearest priori frame of frame carries out the intermediate distance that the true frame is calculated in distance, then seeks the intermediate distance of all true frames
With obtain total distance, make total distance minimum by repeatedly adjusting priori frame size, and the total distance is as current priori frame
Most short total distance when K, i.e., true frame and priori frame overlapping degree highest are counted, the distance calculation formula in clustering is as follows:
Distance=1-IOU
Wherein, IOU indicates the overlapping degree of true frame Yu priori frame, i.e., when true frame is overlapped with priori frame center, the two
The area and the two of coincidence take the ratio of the area of union.Distance is the intermediate distance of true frame；The purpose of clustering
It is adjustment priori frame number K and priori frame size, so that total distance is minimum, i.e., true frame and priori frame overlapping degree highest, this
The positioning accuracy of detection model can be effectively improved.
In the step S4, object space includes dimension of object and object center；Best priori frame is used for target detection
The dimension of object for the object that neural network forecast obtains, which is limited in, to be limited in range, and the restriction range specifically refers to: predetermined times of setting
Number, is expanded and is reduced to each priori frame size according to prearranged multiple, is obtained expanding priori frame and is reduced priori frame, expands
Restriction range of the region as dimension of object between priori frame and diminution priori frame.
The step S6 specifically: in the detection image that vision controller is predicted according to step S5 the type of object into
Row fifth wheel judgement: direct for the object such as hairline, rag remnants, wire, the vision controller that are not belonging to Assembly part
Judge the object for fifth wheel；For belonging to the object of Assembly part, the assembly in current assembly Parts List will not belong to
Part is judged as fifth wheel, the current assembly Parts List be include all assembly zero used in current assembly
The type of part and its corresponding type number.
Beneficial effects of the present invention:
1) present invention obtains information, application scenarios strong applicability by visual sensor；It is obtained by mass data training
Convolutional neural networks extract feature and classification, recognition accuracy is high, and robustness is stronger, and meets real-time detection requirement, can be with
The fifth wheel in region is measured in real time with multi-angle during Automatic manual transmission.
2) present invention energy indirect labor is measured in real time the fifth wheel in specific region in assembling process, has inspection
The advantages that survey accuracy rate is high, strong real-time, using flexible, can be further reduced the introducing of fifth wheel, enhance product reliability
Detailed description of the invention
Fig. 1 is the structural schematic diagram of detection system of the present invention；
Fig. 2 be in the present invention angle adjustable camera structure composition and operation schematic diagram, (a), (b), (c) be respectively three not
With the angle adjustable camera structure schematic diagram of angle；
Fig. 3 is the flow chart of detection method；
Fig. 4 is the flow chart of training Assembly part and common fifth wheel identification model in the present invention.
Wherein, 1- alarm lamp, 2- display screen, the vertical profile of 3-, 4- annular light source and its bracket, 6- control cabinet, 9-
Horizontal profile, 10- angle adjustable camera 3,11- assembly, 12- workbench, 13- vision controller, 14- camera, 15- adjustable angle
Spend camera support, 16- profile, 17- worm screw, 18- worm screw support frame, 20- turbine type camera connection sheet.
Specific embodiment
The present invention will be described in further detail with reference to the accompanying drawings and embodiments.
As shown in Figure 1, the present invention includes workbench 12, angle adjustable camera, annular light source 4, alarm lamp 1, display
Screen 2 and vision controller 13,12 two sides of workbench are fixed with two vertical profiles 3, are connected with level between two vertical profiles 3
Profile 9, assembly 11 are placed in 12 table top of workbench, and annular light source 4, annular light source 4 are provided with right above assembly 11
It is installed in horizontal profile 9 by light source adjusting rod, being provided with positioned at the upper left side, upper right side and surface of assembly 11 can
Hue angle camera, the angle adjustable camera 10 positioned at 11 upper left side of assembly and upper right side are respectively arranged in two vertical profiles 3
On, the angle adjustable camera right above assembly 11 is installed in horizontal profile 9 and is located at right above annular light source 4；Position
In being placed with vision controller 13 in the control cabinet 6 of the lower section of workbench 12, vision controller 13 respectively be installed on above table top
Display screen 2 be connected with the alarm lamp 1 for being installed on the top of workbench 12.Vision controller 13 passes through the display inspection of display screen 2
It surveys as a result, and controlling the alarm of alarm lamp 1 when detecting the fifth wheel in assembly 11.
It is connected between horizontal profile 9 and vertical profile 3 by corner fittings, the freely adjustable height of horizontal profile 9.Annular light source
4, for keeping working region light uniform, eliminate shade, improve the stability of detection system.Ring can be made by adjusting light source adjusting rod
Shape light source 4 is moved freely in horizontal, vertical direction.
As shown in Fig. 2, each angle adjustable camera 10 includes camera 14, camera support 15 and worm screw 17 and turbine type camera
The driving mechanism that connection sheet 20 forms, vertical profile 3 or horizontal profile 9 have been bolted camera support 15, are located at camera
The two sides at 15 back side of bracket are mounted on worm screw support frame 18, and worm screw 17, worm screw 17 are connected between two worm screw support frames 18
Bottom is connected with turbine type camera connection sheet 20, helical tooth and 20 top surface of turbine type camera connection sheet of worm screw 17 by engagement
Gear is meshed；Camera support 15 positioned at 17 lower section of worm screw is provided with arc groove, and turbine type camera connection sheet 20 is by passing through arc
The screw of shape slot is connected with the positive camera 14 of camera support 15 is located at, and turbine type camera connection sheet 20 drives lower edge in worm screw 17
Arc groove rotation realizes the angular adjustment of camera 14 to drive the rotation of camera 14.17 both ends of worm screw are provided with hexagon
Hole rotates the rotation that hexagonal hole realizes worm screw 17 by allen wrench.
Angle adjustable camera can be freely rotated within the scope of larger angle, facilitate adjustment different shooting angles, and worm gear
Worm-gear drive ratio is big, and precision is higher when adjustment camera rotates.
In fifth wheel vision detection system of the invention, annular light source 4 and camera 14 can freely adjust height and angle
Degree facilitates and adapts to different operating demand.Multiple positions of assembly area are arranged in using multiple angle adjustable cameras 10, from difference
Angle acquires assembly area image information in all directions, and detection program is more stringent reliable.
Specific embodiment:
The fifth wheel visible detection method in Automatic manual transmission the following steps are included:
Step 1: opening light source and camera, assembly region when multiple cameras are from multiple angle acquisition Automatic manual transmissions
Picture signal, and the electric signal that picture signal is converted to digital quantity is sent to vision controller.
Step 2: vision controller connects the image that camera acquires a large amount of history assembling process by interface circuit, carries out pre-
The object in image is manually identified after processing operation, and marks type number, centre coordinate, width and the height of each object
Degree constructs data set, then data set is randomly divided into training set and verifying collection in the ratio of 4:1.
The type number of object is labeled according to the type number in object inventory, and object inventory includes visual spatial attention
The type of all objects that can be identified and its corresponding type number, the object that can be identified include Assembly part and are not belonging in device
The fifth wheel of Assembly part；The centre coordinate of object is marked in the coordinate system established using the image upper left corner as origin.
Step 3: clustering being carried out according to the size of training set objects in images, is arranged by clustering best
Priori frame number and best priori frame size；
It first extracts object width and height all in all training set images of step S2 and is used as true frame, then be arranged
K priori frame increases since 1 one by one as cluster centre, priori frame number K, poly- by carrying out Kmeans to all true frames
Alanysis obtains the corresponding most short total distance D of K priori frameK, when K is bigger, DKIt is smaller, but the calculation amount of model predictive process is got over
Greatly.
Meet the priori frame number K of the following conditions as best priori frame number, illustrates that most short total distance variation is slow, this
When K priori frame as best priori frame, the corresponding priori frame size of each best priori frame is as best priori frame size:
||DK-1-DK|-|DK-DK+1| | < minimum threshold of distance
Wherein, K is priori frame number；DKTo carry out Kmeans cluster point to all true frames when priori frame number is K
Analyse obtained most short total distance；
The process for carrying out Kmeans clusterings to all true frames is as follows: will each true frame with it is nearest from the true frame
Priori frame carry out distance the intermediate distance of the true frame be calculated, then the intermediate distance of all true frames is summed to obtain
Total distance makes total distance minimum by repeatedly adjusting priori frame size, and when the total distance is as current priori frame number K
Most short total distance, i.e., true frame and priori frame overlapping degree highest, the distance calculation formula in clustering are as follows:
Distance=1-IOU
Wherein, IOU indicates the overlapping degree of true frame Yu priori frame, i.e., when true frame is overlapped with priori frame center, the two
The area and the two of coincidence take the ratio of the area of union.Distance is the intermediate distance of true frame；The purpose of clustering
It is adjustment priori frame number K and priori frame size, so that total distance is minimum, i.e., true frame and priori frame overlapping degree highest, this
The positioning accuracy of detection model can be effectively improved.
Step 4: YOLOv3 target detection network being trained using training set, obtains Assembly part and common fifth wheel
Identification model, specific steps include:
4.1) the priori frame number clustered in lot number amount, the number of iterations, learning rate and step 3 and size etc. are set
Training parameter；
4.2) stochastic linear scaling, overturning, adjustment brightness, contrast, color are carried out to the image that will input network model
The data enhancement operations such as tune improve the generalization ability of model and the accuracy of identification to wisp；
4.3) YOLOv3 model first uses convolutional neural networks Darknet-53 to extract characteristic pattern, then to various sizes of spy
Sign figure carries out regression analysis, predicts kind of object and position；
4.4) legitimate reading of comparison prediction result and mark calculates penalty values according to loss function；
4.5) model parameter is updated according to the size backpropagation of penalty values；
4.6) step 4.2-4.5 is repeated, reaches maximum number of iterations then deconditioning, by average loss value in whole process
Corresponding model verifies model comprehensive performance on verifying collection as final output model when minimum.
Trained model mainly identifies two type objects:
(1) all Assembly parts that may be used；
(2) the common fifth wheel such as hairline, rag remnants, wire.
In the step S4, object space includes dimension of object and object center；The best priori obtained according to step S3
The size of frame obtains final dimension of object: the object for respectively obtaining model prediction on the size basis of each best priori frame
Body size, which is limited in, to be limited in range, and the dimension of object that would be limited to limit in range exports as a result, by all knots
The highest dimension of object of confidence level is exported as final dimension of object in fruit output；
The restriction range specifically refers to: setting prearranged multiple expands each priori frame size according to prearranged multiple
Big and diminution obtains expanding priori frame and reduces priori frame, expands the region between priori frame and diminution priori frame as object
The restriction range of size.
Step 5: the image that vision controller real-time reception camera is passed back, the assembly zero of input step 4 after being pre-processed to it
Part and common fifth wheel identification model, predict the type and specific location of every objects in images.Using being good at concurrent operation
GPU operational objective detect network, guarantee detection real-time.
Step 6: the kind of object that vision controller is obtained according to prediction judges whether the object belongs to fifth wheel, marks
The position of fifth wheel and alarm out.The judgement of fifth wheel is divided into two steps:
6.1) for the Assembly part identified, vision controller judges each zero according to the Parts List pre-set
Whether part belongs to this assembly, if not then regarding as fifth wheel；The Parts List wherein pre-set includes current
The type for all Assembly parts that assembly uses is numbered
6.2), wire common fifth wheel remaining for the such as hairline, rag that identify, vision controller are directly recognized
The fixed type objects are fifth wheel.
Wherein, the pretreatment in step 2 and step 5 includes:
(1) image is cut, makes picture traverse and height than being 1 to 1；
(2) linear scaling is carried out to image, image is made to become 1024 × 1024 size.
Claims (9)
1. a kind of fifth wheel vision detection system in Automatic manual transmission based on deep learning, which is characterized in that including workbench
(12), angle adjustable camera, annular light source (4), alarm lamp (1), display screen (2) and vision controller (13), workbench
(12) two sides are fixed with two vertical profiles (3), are connected with horizontal profile (9) between two vertical profiles (3), assembly (11)
It is placed in workbench (12) table top, is located at right above assembly (11) and is provided with annular light source (4), annular light source (4) passes through light
Source adjusting rod is installed on horizontal profile (9), is provided with positioned at upper left side, upper right side and the surface of assembly (11) adjustable
Angle camera, the angle adjustable camera (10) for being located at assembly (11) upper left side and upper right side are respectively arranged in two vertical profiles
(3) on, it is located at the angle adjustable camera right above assembly (11) and is installed on horizontal profile (9) and is located at annular light source (4)
Surface；It is placed with vision controller (13) in control cabinet (6) below workbench (12), vision controller (13) is respectively
It is connected with the display screen (2) being installed on above table top and the alarm lamp (1) being installed at the top of workbench (12).
Each angle adjustable camera (10) includes camera (14), camera support (15) and worm screw (17) and turbine type camera connection sheet
(20) driving mechanism formed, vertical profile (3) or horizontal profile (9) have been bolted camera support (15), are located at phase
The two sides at machine support (15) back side are mounted on worm screw support frame (18), are connected with worm screw between two worm screw support frames (18)
(17), worm screw (17) bottom is connected with turbine type camera connection sheet (20), the helical tooth and turbine type of worm screw (17) by engagement
The gear of camera connection sheet (20) top surface is meshed；Camera support (15) below worm screw (17) is provided with arc groove, turbine
Formula camera connection sheet (20) is connected by passing through the screw of arc groove with camera support (15) positive camera (14) is located at, turbine
Formula camera connection sheet (20) is rotated in the case where worm screw (17) drive along arc groove, to drive the rotation of camera (14), realizes phase
The angular adjustment of machine (14).
2. the fifth wheel vision detection system in a kind of Automatic manual transmission based on deep learning according to claim 1, feature
It is, worm screw (17) both ends are provided with hexagonal hole, rotate hexagonal hole by allen wrench and realize worm screw (17)
Rotation.
3. the fifth wheel vision detection system in a kind of Automatic manual transmission based on deep learning according to claim 1, feature
It is, the vision controller (13) shows testing result by display screen (2), and extra in assembly (11) detecting
Alarm lamp (1) alarm is controlled when object.
4. using the detection method of any fifth wheel vision detection system based on deep learning of claim 1-3,
It is characterized in that, comprising the following steps:
S1: opening annular light source (4) and camera (14), assembly when three cameras (14) are from multiple angle acquisition Automatic manual transmissions
(11) image；
S2: all images that vision controller is acquired in history assembling process are pre-processed and are marked, and will be after label
Image be divided into training set and verifying collection；
S3: clustering is carried out according to the size of training set objects in images, best priori frame is arranged by clustering
Number and best priori frame size；
S4: the training set input YOLOv3 target detection network after step S3 clustering is trained, and is collected in verifying
On verified, obtain the Assembly part that can identify kind of object and position and common fifth wheel identification model；
S5: vision controller by from camera real-time reception to detection image pre-process after input step S4 Assembly part and often
See fifth wheel identification model, prediction obtains the type of object and position in detection image；
S6: vision controller judges whether the object in detection image belongs to current assembly and make according to the prediction result of step S5
Part, if not then regarding as fifth wheel, marking the position of fifth wheel and being alarmed by alarm lamp.
5. the detection method of the fifth wheel vision detection system according to claim 4 based on deep learning, feature exist
In the pretreatment operation in the step S2 and step S5 includes carrying out cutting to image and carrying out linear scaling to image.
6. the detection method of the fifth wheel vision detection system according to claim 4 based on deep learning, feature exist
In the step S2 specifically: all images for acquiring vision controller in history assembling process pre-process, and mark
Type number, centre coordinate, width and the height for infusing each object in pretreated image, after pretreatment and mark
All picture construction data sets, then data set is randomly divided into training set and verifying collection in 4 to 1 ratio；
The type number of the object is labeled according to the type number in object inventory, and object inventory includes visual spatial attention
The type of all objects that can be identified and its corresponding type number, the object that can be identified include Assembly part and are not belonging in device
The fifth wheel of Assembly part.
7. the detection method of the fifth wheel vision detection system according to claim 4 based on deep learning, feature exist
In the step S3 specifically: extract object width and height all in all training set images of step S2 as true
Real frame, then K priori frame is set as cluster centre, priori frame number K increases one by one since 1, by all true frames
It carries out Kmeans clustering and obtains the corresponding most short total distance D of K priori frameK；
Meet the priori frame number K of the following conditions as best priori frame number, corresponding K priori frame is as best priori
Frame, the corresponding priori frame size of each most preferably priori frame is as best priori frame size:
||DK-1-DK|-|DK-DK+1| | < minimum threshold of distance
Wherein, K is priori frame number；DKTo carry out Kmeans clustering to all true frames and obtaining when priori frame number is K
The most short total distance arrived.
8. the detection method of the fifth wheel vision detection system according to claim 7 based on deep learning, feature exist
In the process for carrying out Kmeans clusterings to all true frames is as follows: will each true frame with most from the true frame
Close priori frame carries out the intermediate distance that the true frame is calculated in distance, and then the intermediate distance of all true frames is summed
To total distance, make total distance minimum by adjusting priori frame size, and the total distance as current priori frame number K when most
Short total distance, i.e., true frame and priori frame overlapping degree highest, the distance calculation formula in clustering are as follows:
Distance=1-IOU
Wherein, IOU indicates that the overlapping degree of true frame Yu priori frame, distance are the intermediate distance of true frame.
9. the detection method of the fifth wheel vision detection system according to claim 4 based on deep learning, feature exist
In the step S6 specifically: the type of object carries out more in the detection image that vision controller is predicted according to step S5
Excess judgement: object such as hairline, rag remnants, wire, vision controller for being not belonging to Assembly part directly judge
The object is fifth wheel；For belonging to the object of Assembly part, the Assembly part in current assembly Parts List will not belong to
Be judged as fifth wheel, the current assembly Parts List be include all Assembly parts used in current assembly
Type and its corresponding type number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910621434.6A CN110385282B (en) | 2019-07-10 | 2019-07-10 | System and method for visually detecting excess based on deep learning in mechanical assembly |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910621434.6A CN110385282B (en) | 2019-07-10 | 2019-07-10 | System and method for visually detecting excess based on deep learning in mechanical assembly |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110385282A true CN110385282A (en) | 2019-10-29 |
| CN110385282B CN110385282B (en) | 2020-06-16 |
Family
ID=68286403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910621434.6A Expired - Fee Related CN110385282B (en) | 2019-07-10 | 2019-07-10 | System and method for visually detecting excess based on deep learning in mechanical assembly |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110385282B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127558A (en) * | 2019-12-20 | 2020-05-08 | 北京理工大学 | Method and device for determining detection angle of assembly, electronic equipment and storage medium |
| CN111145239A (en) * | 2019-12-30 | 2020-05-12 | 南京航空航天大学 | Aircraft fuel tank redundancy automatic detection method based on deep learning |
| CN112528841A (en) * | 2020-12-11 | 2021-03-19 | 中航贵州飞机有限责任公司 | Intelligent recognition system is used in paperless production based on neural network |
| CN112883842A (en) * | 2021-02-02 | 2021-06-01 | 四川省机械研究设计院(集团)有限公司 | Motorcycle engine assembling method and system based on mutual matching of parts and light source |
| CN113049600A (en) * | 2021-03-31 | 2021-06-29 | 上汽通用五菱汽车股份有限公司 | Engine part wrong and neglected loading detection method and system based on visual detection |
| CN113781407A (en) * | 2021-08-24 | 2021-12-10 | 五邑大学 | Screw detection method, device and computer-readable storage medium |
| CN114713512A (en) * | 2022-03-18 | 2022-07-08 | 安徽信福乡田生态农业有限公司 | Grain color selector and method thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013250809A (en) * | 2012-05-31 | 2013-12-12 | Casio Comput Co Ltd | Multi-class discrimination device, method and program |
| CN105044122A (en) * | 2015-08-25 | 2015-11-11 | 安徽工业大学 | Copper part surface defect visual inspection system and inspection method based on semi-supervised learning model |
| CN109323987A (en) * | 2018-10-17 | 2019-02-12 | 广州超音速自动化科技股份有限公司 | A kind of regulating device of vision detection system |
| CN208555088U (en) * | 2018-06-19 | 2019-03-01 | 苏州振畅耘博智能科技有限公司 | A kind of lithium battery defect inspection machine based on laser and deep learning |
| CN109816725A (en) * | 2019-01-17 | 2019-05-28 | 哈工大机器人（合肥）国际创新研究院 | A kind of monocular camera object pose estimation method and device based on deep learning |
| CN208968931U (en) * | 2018-08-20 | 2019-06-11 | 吉林大学 | Universe deforms online testing device in tire tread rubber material trace |
-
2019
- 2019-07-10 CN CN201910621434.6A patent/CN110385282B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013250809A (en) * | 2012-05-31 | 2013-12-12 | Casio Comput Co Ltd | Multi-class discrimination device, method and program |
| CN105044122A (en) * | 2015-08-25 | 2015-11-11 | 安徽工业大学 | Copper part surface defect visual inspection system and inspection method based on semi-supervised learning model |
| CN208555088U (en) * | 2018-06-19 | 2019-03-01 | 苏州振畅耘博智能科技有限公司 | A kind of lithium battery defect inspection machine based on laser and deep learning |
| CN208968931U (en) * | 2018-08-20 | 2019-06-11 | 吉林大学 | Universe deforms online testing device in tire tread rubber material trace |
| CN109323987A (en) * | 2018-10-17 | 2019-02-12 | 广州超音速自动化科技股份有限公司 | A kind of regulating device of vision detection system |
| CN109816725A (en) * | 2019-01-17 | 2019-05-28 | 哈工大机器人（合肥）国际创新研究院 | A kind of monocular camera object pose estimation method and device based on deep learning |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111127558A (en) * | 2019-12-20 | 2020-05-08 | 北京理工大学 | Method and device for determining detection angle of assembly, electronic equipment and storage medium |
| CN111127558B (en) * | 2019-12-20 | 2023-06-27 | 北京理工大学 | Method and device for determining assembly detection angle, electronic equipment and storage medium |
| CN111145239A (en) * | 2019-12-30 | 2020-05-12 | 南京航空航天大学 | Aircraft fuel tank redundancy automatic detection method based on deep learning |
| CN111145239B (en) * | 2019-12-30 | 2022-02-11 | 南京航空航天大学 | Aircraft fuel tank redundancy automatic detection method based on deep learning |
| CN112528841A (en) * | 2020-12-11 | 2021-03-19 | 中航贵州飞机有限责任公司 | Intelligent recognition system is used in paperless production based on neural network |
| CN112883842A (en) * | 2021-02-02 | 2021-06-01 | 四川省机械研究设计院(集团)有限公司 | Motorcycle engine assembling method and system based on mutual matching of parts and light source |
| CN113049600A (en) * | 2021-03-31 | 2021-06-29 | 上汽通用五菱汽车股份有限公司 | Engine part wrong and neglected loading detection method and system based on visual detection |
| CN113781407A (en) * | 2021-08-24 | 2021-12-10 | 五邑大学 | Screw detection method, device and computer-readable storage medium |
| CN114713512A (en) * | 2022-03-18 | 2022-07-08 | 安徽信福乡田生态农业有限公司 | Grain color selector and method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110385282B (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110385282A (en) | Fifth wheel vision detection system and method in Automatic manual transmission based on deep learning | |
| USRE44353E1 (en) | System and method for assigning analysis parameters to vision detector using a graphical interface | |
| CN112534470B (en) | System and method for image-based target object inspection | |
| CN111144262B (en) | Process anomaly detection method based on monitoring video | |
| CN105473927B (en) | For the apparatus and method for the machine for ensureing automatically working | |
| US8295552B2 (en) | Method for setting parameters of a vision detector using production line information | |
| EP3642696B1 (en) | Method and device for detecting a user input on the basis of a gesture | |
| US20050275831A1 (en) | Method and apparatus for visual detection and inspection of objects | |
| JP5351925B2 (en) | Inspection device and inspection method for long member for transfer mechanism including steel cord | |
| CN111815555A (en) | Metal additive manufacturing image detection method and device combining anti-neural network with local binary | |
| US20180029834A1 (en) | Monitoring of step rollers and maintenance mechanics of passenger conveyors | |
| CN104680145B (en) | The on off state change detecting method and device of a kind of | |
| CN104346802A (en) | Method and device for monitoring off-job behaviors of personnel | |
| CN115184359A (en) | Surface defect detection system and method capable of automatically adjusting parameters | |
| US11145068B2 (en) | Image-processing apparatus, image-processing system, image-processing method, and storage medium | |
| CN105976398A (en) | Daylight fire disaster video detection method | |
| CN110260790A (en) | A kind of detection device and its detection method of the punching angle steel based on 3D vision | |
| CN108972556B (en) | Wire grabbing system and method in complex illumination environment on micro special motor production line | |
| CN109059789A (en) | Cable pitch online test method based on machine vision | |
| CN115144399A (en) | Assembly quality detection method and device based on machine vision | |
| US10939024B2 (en) | Image processing system, image processing device and image processing program for image measurement | |
| CN115258865A (en) | Identification method and device for elevator door | |
| CN112873200A (en) | A degree of depth camera calibration device for industrial robot snatchs field | |
| CN111932581A (en) | Safety rope detection method and device, electronic equipment and readable storage medium | |
| CN117409332B (en) | Long wood shaving appearance data detection system and method based on big data processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200616 |