CN107150032B - Workpiece identification and sorting device and method based on multi-image acquisition equipment - Google Patents
Workpiece identification and sorting device and method based on multi-image acquisition equipment Download PDFInfo
- Publication number
- CN107150032B CN107150032B CN201610125496.4A CN201610125496A CN107150032B CN 107150032 B CN107150032 B CN 107150032B CN 201610125496 A CN201610125496 A CN 201610125496A CN 107150032 B CN107150032 B CN 107150032B
- Authority
- CN
- China
- Prior art keywords
- workpiece
- picking
- image acquisition
- workpieces
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
- B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
- B07C5/3422—Sorting according to other particular properties according to optical properties, e.g. colour using video scanning devices, e.g. TV-cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Manipulator (AREA)
Abstract
The invention relates to a workpiece identification and sorting device based on multiple image acquisition devices, which comprises a first image acquisition device, a second image acquisition device, a workpiece conveying device, a sorting device and a computer, wherein the workpiece conveying device is used for conveying workpieces in one direction; the first image acquisition equipment is positioned at the upstream of the workpiece conveying device and is used for acquiring images of all workpieces positioned at the upstream of the workpiece conveying device; the picking device picks the workpieces on the workpiece conveying device according to the control of the computer; the second image acquisition device is positioned at the downstream of the workpiece conveying device and is used for acquiring local images of the single workpieces picked by the picking device; the computer is used to calculate the pick time, pick position and release position for each workpiece. The invention also relates to a workpiece identification and sorting method based on the multi-image acquisition equipment. The invention can ensure the identification range and the identification precision simultaneously when identifying the workpiece.
Description
Technical Field
The invention relates to the technical field of workpiece identification and sorting, in particular to a workpiece identification and sorting device and method based on multi-image acquisition equipment.
Background
Workpiece sorting, picking and carrying based on machine vision are increasingly widely applied in the production fields of machinery, food, medicine, cosmetics and the like. In these production lines, a plurality of workpieces are mixed and conveyed to a sorting area, and a device for carrying out sorting and conveying is required to be capable of identifying the workpieces in the sorting area, and conveying or placing the workpieces to a required position according to process requirements after picking up the workpieces. The patent "robot sorting system and method based on machine vision" (institute of middle ship's heavy worker 716, CN103706568A) adopts a single camera to acquire images of static circular tablets, and then sorts the tablets according to shapes by a robot, wherein the error range between the suction position and the actual position of the articles is 0.2 mm. The patent "robot automatic palletizing based on vision processing" (shenyang industrial university, CN104058260A) obtains images of static workpieces in a sorting area through a single camera, obtains geometric centroid of the workpieces after image processing, and then performs palletizing by a palletizing robot according to the centroid. The patent "industrial robot sorting system" (shenyang industry university, CN204036474U) adopts the movement of a single camera to acquire images at two different positions, and obtains the geometric centroid and the placing direction of the workpiece by using a binocular stereo vision algorithm to generate the spatial position of the workpiece in the sorting area for the robot to grasp. In order to improve the attitude accuracy of robot feeding, a 2-dimensional camera obtains position compensation data before grabbing a single workpiece at a fixed position, and a 3-dimensional camera obtains attitude characteristic compensation data after picking, the method is mainly used for solving the accuracy problem of feeding and clamping irregular-shaped blank parts.
In the above patent using a single camera, the measurement accuracy obtained by image processing is not high unless expensive high resolution cameras are used and the pixels are sufficiently large in order to obtain multiple images of the workpiece within one field of view.
The application aims at single-layer workpiece occasions, and for a sorting area only accommodating single-layer workpieces to be tiled, because new workpieces continuously flow into some production lines, the workpieces in the sorting area are pushed and extruded mutually, each workpiece can slowly translate and rotate, and the positions of the workpieces have small changes in time slices before image processing, motion track calculation and the like after image acquisition. A slight displacement of the gripper relative to the previous position may also occur during suction or jaw tightening to effect the pick. None of the above patents takes these position changes into account, and the positioning accuracy cannot be ensured.
In addition, the above-mentioned solution using a single camera cannot realize the coordination of multiple picking devices, such as robots. The picking efficiency is low and the use rate of the cameras is low, and the increase of the picking equipment inevitably leads to the multiple increase of the number of the cameras. If a scheme that a single camera images for different identified ranges twice is adopted, not only is the extra cost brought by frequent focal length change of the camera increased, but also the picking efficiency is seriously influenced. In the patent "a vision positioning method for robot handling operation", a first 2D camera is fixed on a robot arm, and the robot picks up a workpiece and then moves to another fixed 3D camera for position compensation and positioning. This approach can also result in a doubling of the number of cameras in multi-robot applications. The efficiency of use of the camera cannot be fully utilized.
Disclosure of Invention
The invention aims to provide a workpiece identification and sorting device and method based on multi-image acquisition equipment, which can ensure the identification range and the identification precision simultaneously when identifying workpieces and solve the problem of the influence of displacement change on the precision caused by the time difference between identification and grabbing of the workpieces continuously flowing into a production line.
The technical scheme adopted by the invention for solving the technical problems is as follows: the workpiece identification and sorting device based on the multiple image acquisition devices comprises a first image acquisition device, a second image acquisition device, a workpiece conveying device, a sorting device and a computer, wherein the workpiece conveying device is used for conveying workpieces in one direction; the first image acquisition equipment is positioned at the upstream of the workpiece conveying device and is used for acquiring images of all workpieces positioned at the upstream of the workpiece conveying device; the picking device picks the workpieces on the workpiece conveying device according to the control of the computer; the second image acquisition device is positioned at the downstream of the workpiece conveying device and is used for acquiring local images of the single workpieces picked by the picking device; the computer is used for calculating the picking time, the picking position and the releasing position of each workpiece and is respectively connected with the first image acquisition device, the second image acquisition device, the workpiece conveying device and the picking device.
The computer calculates the picking time and the picking position of the picking device aiming at each workpiece according to the position of each workpiece when being identified and the moving speed of the workpiece conveying device.
The computer also calculates the workpiece position correction compensation quantity according to the position information of the workpiece positioning characteristics obtained by the second image acquisition equipment, and calculates the release position of the workpiece according to the target position of workpiece release.
The workpiece positioning feature refers to the shape or contour line of the part, which is not shielded by the picking device, on the picked surface of the workpiece.
The picking device picks workpieces in a sucking or grabbing mode.
The first image acquisition device and the second image acquisition device are both industrial cameras. Preferably, the camera resolution is no more than 500 million pixels, or the pixel array is less than or equal to 1920 x 2560.
The workpiece conveying device is also provided with a sensor connected with a computer, and the sensor is used for detecting the conveying position and/or speed of the workpiece conveying device.
The picking devices are more than two, the number of the second image acquisition devices is less than or equal to that of the picking devices, and the picking devices and the second image acquisition devices are arranged at intervals at the downstream of the workpiece conveying device.
The picking devices are more than two, and the picking devices avoid interference collision on a moving path.
The technical scheme adopted by the invention for solving the technical problems is as follows: the method for identifying and sorting the workpieces based on the multi-image acquisition equipment comprises the following steps:
(1) the first image acquisition device acquires images containing a plurality of workpieces at the upstream of the workpiece conveying device, performs image processing, and transmits information for identifying a first position of the workpiece to be picked to the computer;
(2) the computer controls the picking device to be positioned at a second position of the workpiece, and the workpiece is picked when the workpiece reaches the second position;
(3) the second image acquisition equipment acquires images of the positioning features of the workpieces picked by the picking device and obtains position information of the workpiece positioning features relative to the picking device through image processing;
(4) the computer controls the picking device to be positioned to the release position to release the workpiece according to the obtained position information of the workpiece positioning characteristic relative to the picking device.
The method also comprises a step of calculating the picking time and the second position of the picking device for each workpiece according to the first position and the moving speed of the workpiece conveying device when each workpiece is identified by the computer between the step (1) and the step (2).
And (4) calculating the compensation amount of the target position of the workpiece released by the picking device by the computer according to the obtained position information of the workpiece positioning feature relative to the picking device.
The positioning characteristic of the step (3) refers to the shape or contour line of the part which is not shielded by the picking device on the picked surface of the workpiece.
And the step (3) is specifically to carry out image acquisition and gray threshold processing on the picked workpieces, extract contour lines, obtain coordinates of corner points and corners of the contour lines, obtain offset through the coordinates of the corner points and obtain angle offset through the corners of the contour lines.
When the number of the picking devices is more than two, the computer respectively controls the picking devices to be positioned at the second positions of the workpieces in the step (2), the workpieces are picked when the workpieces reach the second positions, and then the picking devices are sequentially moved into the acquisition area of the second image acquisition equipment; multiple picking devices avoid interfering collisions in the path of travel.
In order to increase the efficiency of recognition and sorting, the picking device, which has first finished picking the workpieces, is first moved into the acquisition area of the second image acquisition device.
When the number of the picking devices is more than two, the number of the second image acquisition devices is less than or equal to the number of the picking devices, each second image acquisition device at least corresponds to one picking device, in the step (2), the computer respectively controls the picking devices to be positioned at the second positions of the workpieces, the workpieces are picked when the workpieces reach the second positions, and then the picking devices move into the acquisition areas of the corresponding second image acquisition devices.
Advantageous effects
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects:
the invention can realize high-precision identification and picking of articles in a large range by adopting the multiple cameras, and solves the contradiction between the camera identification range and the camera resolution in the traditional method.
The first camera in the invention obtains the sorting information of the workpieces and the position information which can sort the workpieces. Before picking up the workpiece and completing the positioning measurement, the actual spatial position of the workpiece is different from the spatial position obtained by the identification measurement, including the movement of the workpiece in the period from the identification photographing to the grabbing, and the movement or the deviation of the workpiece before the mechanism is clamped or completely sucked in the grabbing process. Different workpiece positioning characteristics are different, fine positioning measurement is carried out by the second camera according to the positioning characteristics of the grabbed workpiece, high image measurement accuracy can be obtained in imaging of the second camera due to the fact that the size of the workpiece positioning characteristics is small, and meanwhile, due to the fact that relative movement between the picking device and the grabbed workpiece does not exist any more, positioning information of the current workpiece with enough accuracy can be obtained. Therefore, the invention can be applied to sorting, positioning and carrying of workpieces in a sorting area in which a plurality of varieties of workpieces continuously flow in are mixed, thereby solving the problem that the precision is influenced by the displacement change caused by the time difference between recognition and grabbing of the continuously flowing workpieces on a production line and realizing the high-efficiency and high-precision robot article sorting operation.
The cameras used in the present invention are all industrial cameras of common specifications, and there is no particular requirement in terms of resolution and performance. Therefore, the overall cost of the scheme of the invention is lower.
The first camera and the second camera used in the invention are respectively arranged at the upstream and the downstream of the workpiece conveying device, so that the work of the two cameras has no influence on each other, and the continuous and efficient work of mutual matching can be realized. The first camera carries out overall recognition on the workpieces in the workpiece area, so that the overall coordination effect is realized, the picking mechanisms and the second camera at the downstream of the conveying device can pick the workpieces at the same time according to the positions of the workpieces recognized by the first camera, and the working efficiency is improved.
In use cases where multiple picking devices are employed, the multiple picking devices may share the acquisition of the first camera. At the same time, the second camera can be used alternately by a plurality of picking devices with a reasonable layout of the positions of the picking devices. Therefore, the whole device improves the efficiency, minimizes the use number of cameras, improves the utilization rate of the cameras and reduces the implementation cost.
Drawings
FIG. 1 is a schematic diagram of the apparatus of the present invention;
FIG. 2 is a top view of the present invention;
FIG. 3 is a block diagram of the system of the present invention;
FIG. 4 is a flow chart of the operation of the present invention;
FIG. 5 is a schematic diagram of an identification area of a first image capture device in an embodiment of the invention;
FIG. 6 is a schematic diagram of an identification area of a second image capturing device in an embodiment of the present invention;
FIG. 7 is a diagram illustrating a correction compensation amount according to an embodiment of the present invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
The invention relates to a workpiece identification and sorting device based on multiple image acquisition devices, which comprises a first image acquisition device 1, a second image acquisition device 2, a workpiece conveying device 5, a sorting device 3 and a computer 6, wherein the workpiece conveying device 5 is used for conveying workpieces 4 in one direction; the first image acquisition apparatus 1 is located upstream of the workpiece conveyor 5 and is used for acquiring images of all workpieces located upstream of the workpiece conveyor 5; the picking device 3 picks the workpieces 4 on the workpiece conveying device 5 according to the control of the computer; the second image acquisition device 2 is located downstream of the workpiece conveyor 5 and is used for acquiring partial images of the single workpieces picked by the picking device 3; the computer 6 is used for calculating the picking time, picking position and releasing position of each workpiece, and the computer 6 is respectively connected with the first image acquiring device 1, the second image acquiring device 2, the workpiece conveying device 5 and the picking device 3. The workpiece conveying device 5 is also provided with a sensor 7 connected with the computer 6, and the sensor 7 is used for detecting the conveying position and/or speed of the workpiece conveying device. And workpiece stacking areas 8 are also arranged on two sides of the whole workpiece identification and sorting device, and the workpiece stacking areas 8 are release positions of the picking device 3.
The computer can calculate the picking time and the picking position of the picking device aiming at each workpiece according to the position of each workpiece when being identified and the moving speed of the workpiece conveying device; the computer can also calculate the correction compensation quantity of the workpiece position according to the position information of the workpiece positioning characteristics obtained by the second image acquisition equipment, and calculate the release position of the workpiece according to the target position of the workpiece release, so that the high-precision identification and picking of large-range articles are realized. The first image acquisition device and the second image acquisition device may each employ an industrial camera of a common specification.
As can be easily found, the invention can realize high-precision identification and picking of large-range articles by adopting the multi-image acquisition equipment, and solves the contradiction between the camera identification range and the camera resolution in the traditional method. The first image acquisition equipment obtains the sorting information of the workpieces and the position information capable of sorting the workpieces. Before picking the workpiece and completing the positioning measurement, the actual spatial position of the workpiece is different from the spatial position obtained by the identification measurement, and the movement of the workpiece in the period from the acquisition of the identification image to the grabbing of the workpiece and the movement or the deviation of the workpiece before the clamping or the complete suction of the mechanism in the grabbing process are included. Different workpiece positioning characteristics are different, fine positioning measurement is carried out by the second image acquisition equipment according to the positioning characteristics of the grabbed workpiece, high image measurement accuracy can be obtained in imaging of the second image acquisition equipment due to the fact that the size of the workpiece positioning characteristics is small, and meanwhile, due to the fact that relative movement between the picking device and the grabbed workpiece is not carried out any more, positioning information of the current workpiece with enough accuracy can be obtained. Therefore, the invention can be applied to sorting, positioning and carrying of workpieces in a sorting area in which a plurality of varieties of workpieces continuously flow in are mixed, thereby solving the problem that the precision is influenced by the displacement change caused by the time difference between recognition and grabbing of the continuously flowing workpieces on a production line and realizing the high-efficiency and high-precision robot article sorting operation.
The image acquisition equipment used in the invention can be industrial cameras with common specifications, and has no special requirements on resolution and performance. Therefore, the overall cost of the scheme of the invention is lower. The first image acquisition equipment and the second image acquisition equipment used in the invention are respectively arranged at the upstream and the downstream of the workpiece conveying device, so that the work of the two image acquisition equipment has no influence on each other, and the continuous and efficient work of mutual cooperation can be realized. The first image acquisition equipment is used for identifying the workpieces in the workpiece area in a global mode, so that the overall coordination effect is achieved, the picking devices and the second image acquisition equipment on the downstream of the transmission device can pick the workpieces at the same time according to the positions of the workpieces identified by the first image acquisition equipment, and the working efficiency is improved.
The invention also relates to a workpiece identification and sorting method based on multi-image acquisition equipment, which comprises the following steps as shown in figures 3 and 4: the first image acquisition device acquires images containing a plurality of workpieces at the upstream of the workpiece conveying device, performs image processing, and transmits information for identifying a first position of the workpiece to be picked to the computer; the computer controls the picking device to be positioned at a second position of the workpiece, and the workpiece is picked when the workpiece reaches the second position; the second image acquisition equipment acquires images of the positioning features of the workpieces picked by the picking device and obtains position information of the workpiece positioning features relative to the picking device through image processing; the computer controls the picking device to be positioned to the release position to release the workpiece according to the obtained position information of the workpiece positioning characteristic relative to the picking device. And the computer obtains the conveying position and/or the conveying speed of the workpiece conveying device through the sensor so as to complete the whole work flow. The identification area of the first image acquisition device is shown in fig. 5, the dashed box 9 is the identification area of the first image acquisition device, the identification area of the second image acquisition device is shown in fig. 6, the dashed circle 10 is the identification area of the second image acquisition device, and the part of the dashed circle overlapping with the workpiece 4 is the positioning feature 11.
The process is further described below by means of three specific examples.
The first embodiment is as follows:
the workpiece transmission device is a conveyor belt, the picking device is a multi-joint robot, and the workpiece is a box body. The first camera and the second camera are both industrial cameras of common specification, and the resolution ratio of the cameras is 500 ten thousand pixels. The conveyer belt is driven by a rotating motor, a rotating encoder is installed on the motor, the rotating angle of the motor is detected, and the computer calculates the moving distance and speed of the conveyer belt by continuously reading the angle fed back by the encoder.
The first camera is fixedly arranged right above the upstream of the entering direction of the workpieces on the conveyor belt, the lens vertically and downwards points to the sorting area, the second camera is a camera and is fixedly arranged right above the sorting area of the workpieces on the downstream of the conveyor belt, the lens vertically and downwards points to the sorting area,
A) the first camera collects images containing a plurality of workpieces on the upstream of the conveyor belt, processes the images, identifies information of positions of the workpieces to be picked and transmits the information to the computer;
B) the computer calculates the picking time and the picking position of the robot aiming at each workpiece according to the position of each workpiece when being identified and the moving speed of the conveyor belt;
C) the computer controls the robot to be positioned to the picking position to pick the workpiece when the workpiece reaches the picking position, and then the robot moves to the position below the second camera;
D) the second camera carries out image acquisition on the positioning features of the workpieces picked by the robot, and position information of the workpiece positioning features relative to the picking device is obtained through image processing;
E) the computer calculates the compensation amount of the target position of the picking device for releasing the workpiece according to the position information of the workpiece positioning characteristic relative to the picking device obtained in the previous step, so that the release position of the workpiece stacking area to be released by the robot is obtained;
F) the computer controls the robot to be positioned at the release position of the workpiece stacking area to release the workpiece;
G) and C, repeating the step C until all the workpieces are picked.
And D) performing positioning feature recognition on the contour line of one corner of the edge of the upward side of the picked box body by the second camera, performing image acquisition and gray threshold processing on the contour line of one corner of the edge of the box body under a specific illumination condition, extracting the contour line, obtaining coordinates of the corner point and corners of the contour line, and obtaining offset quantities X 'and Y' through the coordinates of the corner point, obtaining an angle offset quantity α 'through the corners of the two contour lines, transmitting the numerical values of X', Y 'and α' to the computer, and calculating corrected workpiece position parameters X, Y and α, wherein a dotted line frame in the step D) is the corrected position of the workpiece.
Example two:
the workpiece transmission device is a conveyor belt, the picking device is two multi-joint robots, and the second cameras are also two and respectively correspond to the positions of the two robots. The workpiece is a box body. The first camera and the second camera are both industrial cameras of common specification, and the pixel array thereof is 1920 × 2560. The conveyer belt is driven by a rotating motor, the rotating speed of the motor is constant, and the computer sets and calculates the moving distance and the moving speed of the conveyer belt according to the rotating speed of the motor.
The first camera is fixedly arranged right above the upstream of the entering direction of the workpieces on the conveyor belt, the lens vertically and downwards points to the sorting area, the second camera is fixedly arranged right above the sorting area of the workpieces on the downstream of the conveyor belt, the lens vertically and downwards points to the sorting area,
the two multi-joint robots and the two second cameras are arranged at a certain distance at the downstream of the conveyor belt and do not interfere with each other,
A) the first camera collects images containing a plurality of workpieces on the upstream of the conveyor belt, processes the images, identifies information of positions of the workpieces to be picked and transmits the information to the computer; the computer calculates the moving distance of the workpiece according to the moving state of the workpiece conveyor belt, and identifies the subsequent workpiece again through the first camera before the workpiece identified by the first camera leaves the first camera image acquisition area along with the workpiece conveyor belt;
B) the computer calculates the picking time and the picking position of the robot aiming at each workpiece according to the position of each workpiece when being identified and the moving speed of the conveyor belt;
C) the computer respectively controls the two robots to be positioned at the picking positions of two different workpieces to pick the workpieces when the workpieces reach the picking positions and then respectively move below the corresponding second cameras;
D) the second camera respectively carries out image acquisition on the positioning characteristics of the workpieces picked by the two robots, and position information of the workpiece positioning characteristics relative to the picking device is obtained through image processing;
E) the computer calculates the compensation amount of the target position of the picking device for releasing the workpiece according to the position information of the workpiece positioning characteristic relative to the picking device obtained in the previous step, so that the release positions of the workpiece stacking areas to be released by the two robots are obtained;
F) the computer respectively controls the two robots to be positioned at the release positions of the workpiece stacking area to release the workpieces;
G) and C, repeating the step C until all the workpieces are picked.
And D, performing image acquisition and gray threshold processing on the contour line of one corner of the edge of the box body under a specific illumination condition, extracting the contour line, obtaining coordinates of the corner point and corners of the contour line, and obtaining offset X 'and Y' through the coordinates of the corner point, obtaining angle offset α 'through the corners of the two contour lines, transmitting the values of X', Y 'and α' to a computer, and calculating corrected workpiece position parameters X, Y and α, wherein a dotted line frame in the step 7 is the position of the workpiece after correction.
Example three:
the workpiece conveying device is a conveying belt, the picking device is two multi-joint robots, and the second camera is one robot. The workpiece is a box body. The first camera and the second camera are both industrial cameras of common specification, and the pixel array thereof is 1920 × 2560. The conveyer belt is driven by a rotating motor, a rotating encoder is installed on the motor, the rotating angle of the motor is detected, and the computer calculates the moving distance and speed of the conveyer belt by continuously reading the angle fed back by the encoder.
The first camera is fixedly arranged right above the upstream of the entering direction of the workpieces on the conveyor belt, the lens vertically and downwards points to the sorting area, the second camera is fixedly arranged right above the sorting area of the workpieces on the downstream of the conveyor belt, the lens vertically and downwards points to the sorting area,
the two multi-joint robots are arranged at a certain distance at the downstream of the conveyor belt and do not interfere with each other, the second camera is positioned between the two robots,
A) the first camera collects images containing a plurality of workpieces on the upstream of the conveyor belt, processes the images, identifies information of positions of the workpieces to be picked and transmits the information to the computer; the computer calculates the moving distance of the workpiece according to the moving state of the workpiece conveyor belt, and identifies the subsequent workpiece again through the first camera before the workpiece identified by the first camera leaves the first camera image acquisition area along with the workpiece conveyor belt;
B) the computer calculates the picking time and the picking position of the robot aiming at each workpiece according to the position of each workpiece when being identified and the moving speed of the conveyor belt;
C) the computer controls the robot to be positioned to the picking position to pick the workpiece when the workpiece reaches the picking position, and then the robot moves to the position below the second camera;
D) the second camera carries out image acquisition on the positioning features of the workpieces picked by the robot, and position information of the workpiece positioning features relative to the picking device is obtained through image processing;
E) the computer calculates the compensation amount of the target position of the picking device for releasing the workpiece according to the position information of the workpiece positioning characteristic relative to the picking device obtained in the previous step, so that the release position of the workpiece stacking area to be released by the robot is obtained;
F) the computer controls the robot to be positioned at the release position of the workpiece stacking area to release the workpiece;
G) and C, repeating the step C until all the workpieces are picked.
In order to improve the efficiency of identification and sorting, the robot which finishes picking the workpieces firstly moves to the lower part of the second camera firstly, then finishes the picking device which picks the workpieces and moves to the lower part of the second camera, and the operation is carried out in sequence. When the first robot finishes the step C, the other robot starts to execute the step C at the same time. The two robots should avoid interference collision on the moving path.
And D, performing image acquisition and gray threshold processing on the contour line of one corner of the edge of the box body under a specific illumination condition, extracting the contour line, obtaining coordinates of the corner point and corners of the contour line, and obtaining offset X 'and Y' through the coordinates of the corner point, obtaining angle offset α 'through the corners of the two contour lines, transmitting the values of X', Y 'and α' to a computer, and calculating corrected workpiece position parameters X, Y and α, wherein a dotted line frame in the step 7 is the position of the workpiece after correction.
It follows that in use situations where multiple picking devices are employed, the multiple picking devices may share the acquisition of the first camera. At the same time, the second camera can be used alternately by a plurality of picking devices with a reasonable layout of the positions of the picking devices. Therefore, the whole device improves the efficiency, minimizes the use number of cameras, improves the utilization rate of the cameras and reduces the implementation cost.
Claims (17)
1. A workpiece recognition and sorting device based on multiple image acquisition devices comprises a first image acquisition device, a second image acquisition device, a workpiece conveying device, a sorting device and a computer, and is characterized in that the workpiece conveying device is used for conveying workpieces in one direction; the first image acquisition device is positioned upstream of the workpiece conveying device and is used for acquiring images of a plurality of workpieces continuously flowing into the workpiece conveying device; the picking device picks the workpieces on the workpiece conveying device according to the control of the computer; the second image acquisition device is positioned at the downstream of the workpiece conveying device and is used for acquiring local images of the single workpieces picked by the picking device; the computer is used for calculating the picking time, the picking position and the releasing position of each workpiece; the computer is respectively connected with the first image acquisition device, the second image acquisition device, the workpiece conveying device and the picking device, and the computer calculates the picking time and the picking position of the picking device for each workpiece according to the position of each workpiece when the workpiece is identified and the moving speed of the workpiece conveying device.
2. The apparatus according to claim 1, wherein the computer further calculates a workpiece position correction compensation amount based on the position information of the workpiece positioning feature obtained by the second image obtaining device, and calculates a release position of the workpiece based on the target position of the release of the workpiece.
3. The apparatus of claim 2, wherein the workpiece positioning feature is a shape or contour of a portion of the picked surface of the workpiece that is not covered by the picking device.
4. The multi-image acquisition device-based workpiece recognition and sorting device of claim 1, wherein the picking device picks up the workpiece by suction or grasping.
5. The multi-image acquisition device based workpiece recognition and sorting apparatus of claim 1, wherein the first and second image acquisition devices are industrial cameras.
6. The multi-image acquisition device based workpiece recognition and sorting apparatus of claim 5, wherein the industrial camera has a resolution of no more than 500 ten thousand pixels or a pixel array of less than or equal to 1920 x 2560.
7. The multi-image acquisition device-based workpiece recognition and sorting apparatus of claim 1, wherein the workpiece conveying device is further provided with a sensor connected with a computer, and the sensor is used for detecting the conveying position and/or speed of the workpiece conveying device.
8. The multi-image capturing device based workpiece recognizing and sorting apparatus as claimed in claim 1, wherein the picking means is two or more, the number of the second image capturing devices is less than or equal to the number of picking means, and the picking means and the second image capturing devices are arranged at intervals downstream of the workpiece conveying means.
9. The multi-image acquisition device based workpiece recognition and sorting apparatus of claim 1, wherein the picking means are two or more, the picking means avoiding interference collision on the moving path.
10. A workpiece identification and sorting method based on multi-image acquisition equipment is characterized by comprising the following steps:
(1) the first image acquisition device acquires images containing a plurality of workpieces at the upstream of the workpiece conveying device, performs image processing, and transmits information for identifying a first position of the workpiece to be picked to the computer;
(2) the computer controls the picking device to be positioned at a second position of the workpiece, and the workpiece is picked when the workpiece reaches the second position;
(3) the second image acquisition equipment acquires images of the positioning features of the workpieces picked by the picking device and obtains position information of the workpiece positioning features relative to the picking device through image processing;
(4) the computer controls the picking device to be positioned to the release position to release the workpiece according to the obtained position information of the workpiece positioning characteristic relative to the picking device.
11. The multi-image acquisition device-based workpiece identifying and sorting method according to claim 10, wherein between the steps (1) and (2), further comprising a step of calculating, by the computer, a picking time and a second position of the picking device for each workpiece according to the first position and the moving speed of the workpiece conveying device when each workpiece is identified.
12. The multi-image acquisition device-based workpiece recognition and sorting method according to claim 10, wherein the computer in step (4) calculates the compensation amount of the target position of the picking device for releasing the workpiece according to the obtained position information of the positioning feature of the workpiece relative to the picking device.
13. The method for workpiece recognition and sorting based on multiple image capturing devices as claimed in claim 10, wherein the positioning feature of step (3) is the shape or contour of the portion of the workpiece on the picked side that is not blocked by the picking device.
14. The method for workpiece recognition and sorting based on multiple image capturing devices as claimed in claim 10, wherein the step (3) is specifically to perform image acquisition and gray threshold processing on the picked workpiece, extract the contour lines, obtain coordinates of the corner points and corners of the contour lines, obtain the offset through the coordinates of the corner points, and obtain the angular offset through the corners of the contour lines.
15. The multi-image capturing device based workpiece identifying and sorting method according to claim 10, wherein when the picking device is two or more, the computer controls the picking devices to be located at the second position of the workpiece in step (2), respectively, and waits for the workpiece to reach the second position to pick the workpiece, and then the picking devices are sequentially moved into the capturing area of the second image capturing device; multiple picking devices avoid interfering collisions in the path of travel.
16. The multi-image capturing device based workpiece identifying and sorting method of claim 15, wherein the picking means that picks up the workpiece first moves into the capturing area of the second image capturing device.
17. The multi-image capturing device based workpiece identifying and sorting method of claim 10, wherein when the picking means are two or more, the number of the second image capturing devices is less than or equal to the number of picking means; each second image acquisition device corresponds to at least one picking device, in the step (2), the computer respectively controls the picking devices to be positioned at the second positions of the workpieces, the workpieces are picked when the workpieces reach the second positions, and then the picking devices are moved into the acquisition areas of the respectively corresponding second image acquisition devices.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610125496.4A CN107150032B (en) | 2016-03-04 | 2016-03-04 | Workpiece identification and sorting device and method based on multi-image acquisition equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610125496.4A CN107150032B (en) | 2016-03-04 | 2016-03-04 | Workpiece identification and sorting device and method based on multi-image acquisition equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107150032A CN107150032A (en) | 2017-09-12 |
| CN107150032B true CN107150032B (en) | 2020-06-23 |
Family
ID=59791877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610125496.4A Active CN107150032B (en) | 2016-03-04 | 2016-03-04 | Workpiece identification and sorting device and method based on multi-image acquisition equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107150032B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107597600B (en) * | 2017-09-26 | 2019-08-30 | 北京京东振世信息技术有限公司 | Sorting system and method for sorting |
| KR102600893B1 (en) * | 2018-04-13 | 2023-11-10 | 삼성디스플레이 주식회사 | Apparatus for transferring panel and method for transferring panel |
| CN108993929A (en) * | 2018-08-01 | 2018-12-14 | 穆科明 | A kind of dual-machine linkage industrial machine vision automatic checkout system |
| DE102018126050A1 (en) * | 2018-10-19 | 2020-04-23 | Fraunhofer-Gesellschaft für die angewandte Forschung e.V. | METHOD FOR SORTING SUPPORTING DISPLAYS OF WORKPIECE REPRESENTATIONS |
| CN109352626B (en) * | 2018-11-05 | 2020-02-11 | 佛山市奇创智能科技有限公司 | Equipment and method for grabbing object by robot |
| CN109513630B (en) * | 2018-11-14 | 2021-06-11 | 深圳蓝胖子机器智能有限公司 | Package sorting system, control method thereof and storage medium |
| CN109513629B (en) * | 2018-11-14 | 2021-06-11 | 深圳蓝胖子机器智能有限公司 | Method, device and computer readable storage medium for sorting packages |
| CN109772733B (en) * | 2019-01-15 | 2020-02-18 | 西安交通大学 | Defect bamboo block detecting and sorting device and method based on vision |
| CN110294316B (en) * | 2019-06-04 | 2021-07-13 | 深圳蓝胖子机器智能有限公司 | Method, system and equipment for grabbing articles on conveyor belt |
| US20220331963A1 (en) * | 2019-09-16 | 2022-10-20 | Abb Schweiz Ag | Method And a Device for Picking and Placing Items |
| CN111167731B (en) * | 2019-10-23 | 2022-11-08 | 武汉库柏特科技有限公司 | Product sorting method, product sorting system and intelligent sorting robot |
| CN111112108B (en) * | 2019-12-30 | 2022-05-13 | 上海交通大学 | Hull plate cutting part sorting equipment and method |
| CN111283685A (en) * | 2020-03-05 | 2020-06-16 | 广州市斯睿特智能科技有限公司 | Vision teaching method of robot based on vision system |
| CN112024426A (en) * | 2020-09-01 | 2020-12-04 | 北京智通云联科技有限公司 | Multi-station visual identification system |
| CN112623759B (en) * | 2020-12-18 | 2023-03-10 | 中国建材国际工程集团有限公司 | Glass stacking scheduling method |
| JP7559580B2 (en) * | 2021-01-28 | 2024-10-02 | セイコーエプソン株式会社 | Method for controlling robot system and robot system |
| CN113814984B (en) * | 2021-10-20 | 2023-04-18 | 环旭电子股份有限公司 | Dynamic image positioning method and system for robot emptying |
| CN113877854A (en) * | 2021-12-08 | 2022-01-04 | 辰星（天津）自动化设备有限公司 | Object sorting method applied to parallel robot |
| CN114932541B (en) * | 2022-06-15 | 2023-07-25 | 中迪机器人(盐城)有限公司 | Robot-based automatic assembly system and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4147930A (en) * | 1975-03-20 | 1979-04-03 | U.S. Philips Corporation | Object location detector |
| CN103691678A (en) * | 2013-12-10 | 2014-04-02 | 殷盛江 | Robot for selecting outer diameter sizes of bearings |
| CN103706575A (en) * | 2013-12-31 | 2014-04-09 | 江苏大学 | Device and method for grading and sorting lenses on line based on two-stage image acquisition |
| CN103764304A (en) * | 2011-04-05 | 2014-04-30 | 泽恩机器人有限公司 | Method for invalidating sensor measurements after a picking action in a robot system |
| CN104511436A (en) * | 2013-09-28 | 2015-04-15 | 沈阳新松机器人自动化股份有限公司 | Express sorting method and system based on robot visual servo technology |
| CN204429717U (en) * | 2014-12-18 | 2015-07-01 | 河南机电高等专科学校 | A kind of bottle cap be applied in bottled drink production equipment detects screening plant |
| CN105251699A (en) * | 2015-11-10 | 2016-01-20 | 广东溢达纺织有限公司 | Checking and sorting device for sedimentation patches |
-
2016
- 2016-03-04 CN CN201610125496.4A patent/CN107150032B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4147930A (en) * | 1975-03-20 | 1979-04-03 | U.S. Philips Corporation | Object location detector |
| CN103764304A (en) * | 2011-04-05 | 2014-04-30 | 泽恩机器人有限公司 | Method for invalidating sensor measurements after a picking action in a robot system |
| CN104511436A (en) * | 2013-09-28 | 2015-04-15 | 沈阳新松机器人自动化股份有限公司 | Express sorting method and system based on robot visual servo technology |
| CN103691678A (en) * | 2013-12-10 | 2014-04-02 | 殷盛江 | Robot for selecting outer diameter sizes of bearings |
| CN103706575A (en) * | 2013-12-31 | 2014-04-09 | 江苏大学 | Device and method for grading and sorting lenses on line based on two-stage image acquisition |
| CN204429717U (en) * | 2014-12-18 | 2015-07-01 | 河南机电高等专科学校 | A kind of bottle cap be applied in bottled drink production equipment detects screening plant |
| CN105251699A (en) * | 2015-11-10 | 2016-01-20 | 广东溢达纺织有限公司 | Checking and sorting device for sedimentation patches |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107150032A (en) | 2017-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107150032B (en) | Workpiece identification and sorting device and method based on multi-image acquisition equipment | |
| CN110509281B (en) | Binocular vision-based pose recognition and grabbing device and method | |
| CN109230580B (en) | Unstacking robot system and unstacking robot method based on mixed material information acquisition | |
| US10442633B2 (en) | Robot system | |
| US10232512B2 (en) | Coordinate system setting method, coordinate system setting apparatus, and robot system provided with coordinate system setting apparatus | |
| CN109249390B (en) | Robot system | |
| CN111452040B (en) | System and method for associating machine vision coordinate space in a pilot assembly environment | |
| CN108372503B (en) | Article transport device using at least one sensor | |
| CN110580725A (en) | Box sorting method and system based on RGB-D camera | |
| US20160001983A1 (en) | Article alignment apparatus for aligning article on conveyor | |
| JP5288908B2 (en) | Article alignment system and robot hand | |
| US20150290805A1 (en) | Vision-Assisted System and Method for Picking of Rubber Bales in a Bin | |
| CN105690393A (en) | Four-axle parallel robot sorting system based on machine vision and sorting method thereof | |
| CN104511436A (en) | Express sorting method and system based on robot visual servo technology | |
| CN110980276B (en) | Method for implementing automatic casting blanking by three-dimensional vision in cooperation with robot | |
| EP4032663A1 (en) | Work robot and work system | |
| CN110977939B (en) | Target workpiece identification and positioning system | |
| CN109916346B (en) | Workpiece flatness detection device and method based on vision system | |
| CN111069073A (en) | Rubber ring sorting equipment | |
| CN114632722B (en) | Pickup device | |
| CN208744837U (en) | Intelligent machine arm system based on three-dimensional machine vision | |
| JP6697510B2 (en) | Robot system | |
| CN212238247U (en) | Robot sorting system with flexible feeding mechanism | |
| CN205552536U (en) | Four -axis parallel robot letter sorting system based on machine vision | |
| EP4395965A1 (en) | Robotic system with overlap processing mechanism and methods for operating the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |