CN113752268A - Circuit board grabbing control method, device, equipment and storage medium - Google Patents
Circuit board grabbing control method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113752268A CN113752268A CN202111331010.XA CN202111331010A CN113752268A CN 113752268 A CN113752268 A CN 113752268A CN 202111331010 A CN202111331010 A CN 202111331010A CN 113752268 A CN113752268 A CN 113752268A
- Authority
- CN
- China
- Prior art keywords
- circuit board
- moments
- pose
- calculating
- deviation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1661—Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G47/00—Article or material-handling devices associated with conveyors; Methods employing such devices
- B65G47/74—Feeding, transfer, or discharging devices of particular kinds or types
- B65G47/90—Devices for picking-up and depositing articles or materials
Abstract
The application relates to the technical field of circuit board detection, and particularly discloses a circuit board grabbing control method, device, equipment and storage medium. The method comprises the following steps: acquiring shot images formed by shooting the circuit board at two moments; calculating pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments; calculating the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments; judging whether the pose deviation is smaller than a preset deviation or not; and controlling a mechanical arm clamping jaw to grab the circuit board in response to the pose deviation being smaller than the preset deviation. Compared with a method for grabbing the circuit board under the condition that the circuit board moves, the circuit board grabbing control method provided by the application can grab the circuit board more accurately.
Description
Technical Field
The application relates to the technical field of circuit board detection, in particular to a circuit board grabbing control method, device, equipment and storage medium.
Background
In the related art, the assembled circuit board is transferred to the vicinity of the inspection station via a belt transmission line, and a photographing camera in the industrial robot is triggered to photograph an image. And then, the industrial robot determines the pose of the circuit board based on the shot image and controls the mechanical arm clamping jaw to grab the circuit board based on the pose of the circuit board.
However, in the related art, when the circuit board is grabbed by the mechanical arm clamping jaw, the circuit board still moves along with the belt transmission line, so that the mechanical arm clamping jaw cannot accurately grab the circuit board.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the application provides a circuit board grabbing control method, a device, equipment and a storage medium.
On one hand, the application provides a circuit board grabbing control method which is used for controlling a mechanical arm clamping jaw to grab a circuit board on a belt transmission line; the method comprises the following steps:
acquiring shot images formed by shooting the circuit board at two moments;
calculating pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments;
calculating the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments;
judging whether the pose deviation is smaller than a preset deviation or not;
and controlling a mechanical arm clamping jaw to grab the circuit board in response to the pose deviation being smaller than the preset deviation.
Optionally, the captured image comprises a depth image;
the calculating of the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments comprises:
generating a corresponding source point cloud based on the depth images shot at two moments;
acquiring registration point clouds which are registered with target point clouds in the source point clouds corresponding to the two moments, wherein the target point clouds are acquired in advance and represent point clouds of similar circuit board outlines;
and calculating pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
Optionally, the captured image further comprises a planar image;
the calculating the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments further comprises:
acquiring sub-image areas matched with a target plane image in the plane images corresponding to two moments, wherein the target plane image is a plane image which is acquired in advance and represents the appearance of the same type of circuit board;
determining a corresponding sub-depth image in the depth image based on the sub-image regions corresponding to the two moments;
generating a corresponding source point cloud based on the depth images corresponding to the two moments, including: and generating the corresponding source point cloud based on the sub-depth images in the depth images corresponding to the two moments.
Optionally, the calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments includes:
respectively determining the position coordinates of the set position in the circuit board at two moments based on the registration point clouds corresponding to the two moments;
the calculating of the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments comprises:
and calculating the position deviation based on the position coordinates of the preset position in the circuit board at two moments.
Optionally, the calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments includes:
respectively calculating the principal component directions of the minimum directed bounding boxes which surround the circuit board at two moments based on the registration point clouds corresponding to the two moments;
the calculating of the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments comprises:
and calculating the angle deviation of the circuit board at two moments based on the principal component directions of the minimum directed bounding boxes corresponding to the two moments.
Optionally, the controlling the robot arm gripper to grip the circuit board comprises:
acquiring grabbing offset parameters of the circuit board;
calculating the grabbing pose of the mechanical arm when the mechanical arm grabs the circuit board based on the grabbing bias parameters of the circuit board and the pose parameters of the circuit board at the later moment of the two moments;
and controlling the mechanical arm clamping jaw to clamp the circuit board based on the clamping pose.
On the other hand, the application provides a circuit board grabbing control device, which is used for controlling a mechanical arm clamping jaw to grab a circuit board on a belt transmission line; the device comprises:
the image acquisition unit is used for acquiring shot images formed by the shooting circuit boards at two moments;
the pose parameter determining unit is used for calculating pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments;
the pose deviation determining unit is used for calculating pose deviations of the circuit board at two moments based on pose parameters of the circuit board at two moments;
the judging unit is used for judging whether the pose deviation is within a preset deviation or not;
and the grabbing control unit is used for controlling the clamping jaw of the mechanical arm to grab the circuit board under the condition that the judgment unit judges that the pose deviation is within the preset deviation.
Optionally, the captured image comprises a depth image; the pose parameter determination unit includes:
a source point cloud determination subunit, configured to generate corresponding source point clouds based on the depth images captured at two moments, respectively;
the registration point cloud determining subunit is used for acquiring registration point clouds which are registered with target point clouds in the source point clouds corresponding to the two moments, wherein the target point clouds are acquired in advance and represent point clouds of similar circuit board outlines;
and the pose parameter determining subunit is used for calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
In yet another aspect, the present application provides a computing device comprising: the circuit board grabbing control method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the circuit board grabbing control method is realized.
In still another aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores computer instructions for causing the computer to execute the circuit board capture control method as described above.
Compared with the prior art, the technical scheme provided by the application has the following advantages:
the technical scheme is that the position and pose parameters of the circuit board at two moments are calculated respectively based on shot images formed by shooting the circuit board at two moments, and the position and pose deviation is calculated based on the position and pose parameters at two moments. And when the pose deviation is judged to be smaller than the preset deviation, determining that the circuit board is stable, and then controlling the mechanical arm clamping jaw to grab the circuit board. Compared with a method for grabbing the circuit board under the condition that the circuit board moves, the circuit board grabbing control method provided by the application can grab the circuit board more accurately.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those skilled in the art that other drawings can be obtained from these drawings without inventive exercise, wherein:
FIG. 1 is a schematic structural diagram of a detection station area of a circuit board production line;
fig. 2 is a flowchart of a circuit board grabbing control method provided in the embodiment of the present application;
fig. 3 is a schematic structural diagram of a circuit board capture control apparatus according to some embodiments of the present application;
fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustration purposes only and are not intended to limit the scope of the present application.
The embodiment of the application provides a circuit board grabbing control method, which is used for controlling a mechanical arm clamping jaw to grab a circuit board after the circuit board is determined to be static, and then the circuit board grabbing accuracy is improved.
In order to more clearly understand the circuit board grabbing control method provided by the embodiment of the present application, before describing the circuit board grabbing control method provided by the embodiment of the present application, first, a structure of a production detection station in a circuit board production line is described.
FIG. 1 is a schematic structural diagram of a detection station area of a circuit board production line. As shown in fig. 1, the circuit board production line inspection station area includes a belt transfer line 11 (for convenience, only a portion of the belt transfer line 11 is shown in fig. 1), a blocking device 12, a supply material detection sensor (not shown in the drawings), an industrial robot 13, and an inspection table 14.
The belt conveyor 11 is used to convey the assembled circuit boards upstream of the circuit board production line to the inspection station area.
The blocking device 12 is disposed above the belt transmission line 11 to block the circuit board transmitted through the belt transmission line 11 such that the circuit board is decelerated and stays at one side of the blocking rail. In a particular embodiment, the arresting means 12 may be an arresting rail.
The incoming material detection sensor is arranged on the incoming material side of the blocking cross rod and used for detecting whether a circuit board exists on the incoming material side of the blocking cross rod or not. When it is detected that the incoming material side of the blocking device has a circuit board, the incoming material detection sensor generates a detection signal and sends the detection signal to the industrial robot 13, so that the industrial robot 13 performs circuit board pose detection and circuit board grabbing operation. In particular embodiments, the incoming material detection sensor may be a correlation sensor, or may be another type of sensor.
The industrial robot 13 is used to perform posture detection and grasping operation of the circuit board. Specifically, the industrial robot 13 includes a photographing camera 131, a computer device (not shown in the drawings), a robot arm 132, and a gripper 133 mounted on a distal end of the robot arm 132. The shooting camera is used for shooting the circuit board intercepted by the blocking device to form a shot image, and the shot image is sent to the computer equipment. The computer device determines the pose parameters of the circuit board based on the captured image, generates a control instruction for controlling the motions of the robot arm 132 and the end holding jaw 133 of the robot arm 132 according to the pose parameters of the circuit board, and controls the robot arm 132 and the end holding jaw 133 of the robot arm 132 to grip the circuit board.
As shown in fig. 1, in some embodiments of the present application, the camera is mounted at the end of the robot arm 132, and moves in position as the robot arm 132 moves. In other embodiments of the present application, the camera 131 may be provided independently of the robot arm 132, and may be fixed to an upper side region of the blocking device 12 of the belt conveying line 11, for example.
Fig. 2 is a flowchart of a circuit board grabbing control method provided in the embodiment of the present application. As shown in fig. 2, in the embodiment of the present application, the circuit board grabbing control method may include steps S201 to S205. It should be noted that the circuit board grabbing control method provided by the embodiment of the application is executed by a computer device in an industrial robot.
S201: and acquiring shot images formed by the shooting circuit board at two moments.
In the embodiment of the application, after the circuit board is arranged on the incoming material side of the blocking device, the shooting camera shoots the circuit board blocked by the blocking device to form a shot image, and the shot image is sent to the computer equipment. In the embodiment of the application, the shooting camera shoots the circuit board according to the set shooting frequency to form a shot image, and sends the shot image to the computer equipment.
And after receiving shot images formed by shooting the circuit board intercepted by the blocking device at a plurality of moments by the shooting camera, the computer equipment acquires the shot images corresponding to the two moments according to a set sampling method. For example, the computer device may sample the captured images transmitted from the camera to obtain the captured images corresponding to two times, in such a manner that one captured image is obtained every second.
It should be noted that, in the embodiment of the present application, if the photographing camera is mounted at the end of the robot arm, the computer device first controls the robot arm to move so that the photographing camera moves to the right upper side of the belt transmission line. Subsequently, the computer device controls the photographing camera to photograph the circuit board to obtain a photographed image.
S202: and calculating the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments.
The pose parameters of the circuit board are parameters representing the position and the posture of the circuit board in a certain coordinate system, and the pose parameters of the circuit board can comprise position parameters and posture parameters.
The position parameter of the circuit board is a position coordinate of a specific part of the circuit board, and may be, for example, a position coordinate of a center point or a corner point of the circuit board, or a position coordinate of a component having a specific color or a specific shape in the circuit board.
The attitude parameter of the circuit board is the deflection angle of the circuit board relative to a reference coordinate axis under a certain coordinate system.
It should be noted that the aforementioned coordinate system may be a camera coordinate system, or a world coordinate system determined by an industrial robot coordinate origin, and the embodiment of the present application is not particularly limited.
After the computer equipment acquires the shot images corresponding to the two moments, the two shot images are processed by adopting a corresponding image processing method, and pose parameters of the circuit board at the two moments are obtained through calculation. In specific implementation, the image processing method adopted by the computer equipment is different based on different types of the shot images.
In some embodiments of the present application, the photographing camera is a normal camera (i.e., a 2D camera for general use). The photographed image is a plane image. In this case, the computer device calculating the pose parameters of the circuit board at the two times based on the captured images corresponding to the two times may include the following steps. Firstly, identifying areas of the circuit board in the shot images corresponding to two moments, and then performing inverse perspective transformation on the areas of the circuit board in the shot images corresponding to the two moments by adopting an inverse perspective transformation method based on internal parameters and external parameters of a common camera to obtain inverse perspective images; and then determining the pose parameters of the circuit board at two moments based on the inverse perspective images.
In other embodiments of the present application, the capture camera is a depth camera, which may be, for example, a structured light depth camera, a binocular vision camera, or a light time of flight camera. In the case where the photographic camera is a depth camera, the photographic image includes a depth image. Correspondingly, the step S202 of calculating the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments may include steps S2021 to S2023.
Step S2021: and generating corresponding source point clouds based on the depth images shot at two moments.
In the embodiment of the application, computer equipment firstly obtains an internal reference matrix of a depth camera, and then converts pixel coordinates of each pixel point in a depth image based on the internal reference matrix to obtain source point clouds corresponding to two moments.
Step S2022: and acquiring a registration point cloud which is registered with the target point cloud from the source point cloud corresponding to the two moments.
The target point cloud is pre-stored and represents the point cloud of the outline of the similar circuit board. In the embodiment of the application, the target point cloud may be a point cloud formed by shooting a standard circuit board by using a depth camera.
In this embodiment of the application, for example, an Iterative closest Point algorithm (ICP) or a Normal Distribution Transform (NDT) may be used to determine a registration Point cloud registered with a target power source in source Point clouds corresponding to two time instants.
Step S2023: and calculating pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
After the registration point clouds at two moments are obtained, the appearance outlines of the circuit board at the two moments are obtained. And then calculating pose parameters of the circuit board at two moments based on the registration point cloud.
In the embodiment of the application, the calculation of the pose parameters of the circuit board at two moments based on the registration point cloud comprises the acquisition of position parameters and posture parameters.
In some embodiments of the present application, determining the position parameters of the circuit board at two time instants based on the registration point clouds corresponding to the two time instants may include step a: and determining the position coordinates of the set position in the circuit board at two moments based on the registration point clouds corresponding to the two moments. Specifically, a point representing the set position of the circuit board or a point closest to the set position may be determined according to the phase position relationship of each point in the registration point cloud, and then the position coordinate of the set position may be determined according to the coordinates of the points.
In some embodiments of the present application, determining the pose parameters of the circuit board at two times based on the registration point clouds corresponding to the two times may include step B: and respectively calculating the principal component direction of the minimum directed bounding box which surrounds the circuit board at two moments, and adopting the principal component direction of the minimum bounding box as the attitude parameter of the circuit board. In a specific embodiment, a principal component analysis method may be adopted to obtain principal component directions of the minimum oriented bounding box that bounds the circuit board at two moments, and the principal component directions of the minimum bounding box are used as attitude parameters of the circuit board.
In this embodiment of the application, if the source point clouds corresponding to the depth images at two moments are generated directly in the foregoing step S2021, the source point clouds include source point clouds corresponding to all coordinate points in the depth images. At this time, determining the registration point cloud based on the source point cloud and the target point cloud in the subsequent step S2022 may consume a large amount of computational resources, and may also cause a problem of an error in the obtained registration point cloud due to local matching.
To solve the foregoing problems, in some embodiments of the present application, the captured image captured by the capturing camera includes a planar image in addition to the aforementioned depth image, and the planar image may be captured by a planar camera associated with the coordinates of the depth camera or by a planar lens in the depth camera. In this case, steps S2024-S2025 may also be performed before step S2021 is performed.
Step S2024: and acquiring sub-image areas matched with the target plane image in the plane images corresponding to the two moments.
The target plane image is pre-stored and represents a two-dimensional image of the outline of the same type of circuit board.
After obtaining the planar images corresponding to the two moments, an algorithm such as an average absolute difference algorithm, an absolute error sum algorithm, an error sum of squares algorithm, an average error sum of squares algorithm, a normalized product correlation algorithm, and the like may be used to determine the sub-image region matching the target planar image based on the planar image and the target planar image.
Step S2025: and determining a sub-depth image in the corresponding depth image based on the sub-image regions corresponding to the two moments.
In the embodiment of the application, the coordinates of the plane camera and the depth camera for shooting the plane image are obtained by correlation shooting, or the coordinates of the plane camera and the depth camera are obtained by adopting a plane lens in the depth camera, so that the coordinate conversion relation between the pixel points in the plane image and the pixel points in the depth image is known. After the coordinate transformation relationship is known and the sub-image regions corresponding to the two time instants are obtained, the sub-depth image in the corresponding depth image can be determined based on the sub-image regions.
Correspondingly, the foregoing step S2021 may include: and generating a corresponding source point cloud based on the sub-depth images in the depth images corresponding to the two moments.
After the sub-depth images of the depth images corresponding to the two moments are determined in the steps S2024 and S2025, the corresponding source point clouds are generated only for the sub-depth images, so that the number of points included in the source point clouds can be reduced, and then, the computational power consumption caused by the overlarge data amount of the source point clouds when the registration point clouds are determined based on the source point clouds and the target point clouds in the step S2022 is reduced, and the problem of error of the obtained registration point clouds caused by local matching is avoided.
Step S203: and calculating the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments.
As before, the pose parameters of the circuit board include position parameters and pose parameters. In the embodiment of the application, calculating the pose deviation of the circuit board at two moments can include calculating the position deviation and the attitude deviation.
In some embodiments of the present application, in the case of determining the position coordinates of the set position in the circuit board at two times using the foregoing step a, the step S203 of calculating the position deviation at two times includes the step S2031 of: and calculating the position deviation based on the position coordinates of the preset position in the circuit board at two moments. Specifically, if the position coordinates of the set position in the circuit board at two moments are respectively
And
the positional deviation is
。
In some embodiments of the present application, in the case of adopting the principal component direction of the minimum bounding box as the attitude parameter of the circuit board, the step S203 of calculating the positional deviation at two times includes the step S2032 of: and calculating the angle deviation of the circuit board at two moments based on the principal component directions of the minimum directed bounding boxes corresponding to the two moments. Specifically, if the principal component directions of the minimum bounding boxes corresponding to the two moments are respectively
And
an angular deviation of
。
After determining the pose deviation of the circuit board at two moments, step S204 may be performed.
Step S204: and judging whether the pose deviation is smaller than a preset deviation. If yes, go to step S205; if not, step S201 is executed again.
And judging whether the pose deviation is smaller than a preset deviation, wherein the judgment includes judging whether the position deviation and the angle deviation are smaller than the preset deviation, and the preset deviation includes a preset position deviation and a preset angle deviation. In some embodiments of the present application, the preset positional deviation is 1mm, the preset angular deviation is 1 °, and if the positional deviation is less than 1mm and the angular deviation is less than 1 °, it is determined that the pose deviation of the circuit board is less than the preset deviation.
Step S205: and controlling the mechanical arm clamping jaw to grab the circuit board in response to the situation that the pose deviation is smaller than the preset deviation.
In the embodiment of the application, if the pose deviation is smaller than the preset deviation, it is determined that the position of the circuit board is stable after being intercepted by the blocking device. At the moment, the central control machine controls the mechanical arm clamping jaw to grab the circuit board.
According to the circuit board grabbing control method provided by the embodiment of the application, the position and attitude parameters of the circuit board at two moments are respectively calculated based on the shot images formed by shooting the circuit board at two moments, and the position and attitude deviation is calculated based on the position and attitude parameters at two moments. And when the pose deviation is judged to be smaller than the preset deviation, determining that the circuit board is stable, and then controlling the mechanical arm clamping jaw to grab the circuit board. Compared with a method for grabbing the circuit board under the condition that the circuit board moves, the circuit board grabbing control method provided by the embodiment of the application can grab the circuit board more accurately.
In some embodiments of the present application, the step of controlling the robot arm clamping jaw to clamp the circuit board in step S205 may specifically include steps S2051 to S2053.
Step S2051: and acquiring grabbing offset parameters of the circuit board.
The grabbing offset parameter of the circuit board is an offset parameter representing the grabbing position of the mechanical arm clamping jaw and the pose parameter of the grabbing relative to the circuit board. In some embodiments of the present application, the grab bias parameter may be a grab bias matrix representation.
Step S2052: and based on the grabbing bias parameters of the circuit board and the pose parameters of the circuit board at the later moment, the grabbing pose of the mechanical arm of the computer equipment when the mechanical arm grabs the circuit board is calculated.
In some embodiments of the application, under the condition that the pose parameter of the circuit board is the pose parameter in the world coordinate system, the grabbing pose when the mechanical arm grabs the circuit board can be obtained by directly calculating the grabbing offset parameter and the position parameter of the circuit board. Specifically, the grabbing pose h (grappper position) may be obtained by using h (grappper position) = h (pchandle in base) × t (offset), where h (pchandle in base) is a pose parameter of the circuit board in an actual packaging system, and t (offset) is a conversion matrix determined based on the grabbing offset parameter.
In other embodiments of the present application, in a case that the pose parameter of the circuit board is a pose parameter in a camera coordinate system, the pose parameter in the camera coordinate system needs to be converted into a pose parameter in a world coordinate system, and a capture pose is calculated based on the pose parameter in the world coordinate system and the residence bias parameter. For example, h (grapper position) = t (camera in base) × h (pcbpose in camera) × t (offset) may be adopted to obtain the grapper position h (grapper position), where t (camera in base) is a conversion matrix of the camera coordinate system and the world coordinate system, h (pcpos in camera) is a position parameter of the circuit board in the camera coordinate system, and t (offset) is a conversion matrix determined based on the grapper offset parameter.
Besides providing the circuit board grabbing control method, an embodiment of the present application further provides a circuit board grabbing control device, which is used for controlling a mechanical arm clamping jaw to grab a circuit board on a belt transmission line, where the circuit board is intercepted by a blocking device. Fig. 3 is a schematic structural diagram of a circuit board capture control device according to some embodiments of the present application. The circuit board grabbing control device provided by the embodiment of the application can be a functional module of the computer equipment.
As shown in fig. 3, a circuit board capture control apparatus 300 according to an embodiment of the present application includes an image acquisition unit 301, a pose parameter determination unit 302, a pose deviation determination unit 303, a determination unit 304, and a capture control unit 305
The image acquisition unit 301 is configured to acquire captured images formed by the capturing circuit board at two times.
The pose parameter determination unit 302 is configured to calculate pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments.
The pose deviation determination unit 303 is configured to calculate the pose deviations of the circuit board at two moments based on the pose parameters of the circuit board at two moments.
The determination unit 304 is configured to determine whether the pose deviation is within a preset deviation.
The grasping control unit 305 is configured to control the robot arm gripping claw to grasp the circuit board in the case where the determination unit 304 determines that the posture deviation is within the preset deviation.
In some embodiments of the present application, the captured image comprises a depth image. In this case, the pose parameter determination unit 302 may include a source point cloud determination subunit, a registration point cloud determination subunit, and a pose parameter determination subunit.
The source point cloud determining subunit is used for generating corresponding source point clouds based on the depth images shot at two moments.
The registration point cloud determining subunit is used for acquiring registration point clouds which are registered with target point clouds in source point clouds corresponding to two moments, wherein the target point clouds are acquired in advance and represent point clouds of similar circuit board outlines.
The pose parameter determining subunit is used for calculating pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
In some embodiments of the present application, the captured image may include a planar image in addition to the depth image. Correspondingly, the pose parameter determination unit 302 may further include a sub-image region determination subunit and a sub-depth image determination subunit.
And the sub-image area determining subunit acquires sub-image areas matched with the target plane image from the plane images corresponding to the two moments, wherein the target plane image is acquired in advance and represents the plane image of the appearance of the similar circuit board.
And the sub-depth image determining sub-unit is used for determining the sub-depth image in the corresponding depth image based on the sub-image areas corresponding to the two moments.
Correspondingly, the determining of the source point cloud by the sub-unit based on the depth images corresponding to the two moments generates the corresponding source point cloud specifically as follows: and generating a corresponding source point cloud based on the sub-depth images in the depth images corresponding to the two moments.
In some embodiments of the present application, the pose parameter determination unit 302 calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments includes: and determining the position coordinates of the set position in the circuit board at two moments based on the registration point clouds corresponding to the two moments. Correspondingly, the pose deviation determination unit 303 calculates the position deviation based on the position coordinates of the preset position in the circuit board at two moments.
In some embodiments of the present application, the pose parameter determination unit 302 calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments includes: and respectively calculating the principal component directions of the minimum directed bounding boxes which surround the circuit board at the two moments based on the registration point clouds corresponding to the two moments. Correspondingly, the pose deviation determination unit 303 calculates the angle deviation of the circuit board at two moments based on the principal component directions of the minimum directed bounding boxes corresponding to the two moments.
In some embodiments of the present application, the grasp control unit 305 includes a bias parameter acquisition subunit, a grasp pose determination subunit, and a grasp control subunit. The offset parameter acquiring subunit is used for acquiring the grabbing offset parameters of the circuit board. The grabbing pose determining subunit is used for determining the grabbing pose when the mechanical arm of the computer equipment grabs the circuit board based on the grabbing offset parameter of the circuit board and the pose parameter of the circuit board at the later moment of the two moments. And the grabbing control subunit is used for controlling the mechanical arm clamping jaw to grab the circuit board based on the grabbing pose.
The embodiment of the present application further provides a computer device, which includes a processor and a memory, where the memory stores a computer program, and when the computer program is executed by the processor, the circuit board capture control method according to any of the above embodiments can be implemented.
For example, fig. 4 is a schematic structural diagram of a computer device provided in an embodiment of the present application. The computer device provided by the embodiment of the application is deployed in the edge computing cluster and is in communication connection with the computing nodes in the edge computing cluster through the private network.
Referring now in particular to fig. 4, there is shown a schematic block diagram of a computer device 400 suitable for use in implementing embodiments of the present application. The computer device shown in fig. 4 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present application.
As shown in fig. 4, the computer device 400 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 401 that may perform various appropriate actions and processes in accordance with a program stored in a read only memory ROM402 or a program loaded from a storage means 408 into a random access memory RAM 403. In the RAM403, various programs and data necessary for the operation of the computer apparatus 400 are also stored. The processing device 401, the ROM402, and the RAM403 are connected to each other via a bus 404. An input/output I/O interface 405 is also connected to bus 404.
Generally, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 408 including, for example, tape, hard disk, etc.; and a communication device 409. The communication means 409 may allow the computer device 400 to communicate with other devices, either wirelessly or by wire, to exchange data. While fig. 4 illustrates a computer device 400 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 409, or from the storage device 408, or from the ROM 402. The computer program, when executed by the processing device 401, performs the above-described functions defined in the methods of the embodiments of the present application.
It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the client, computer device may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the computer device; or may exist separately and not be incorporated into the computer device.
The computer readable medium carries one or more programs which, when executed by the computing device, cause the computing device to: receiving a first image downloading request sent by the computing node, wherein the first image downloading request comprises an identifier of an image to be loaded; inquiring whether the mirror image to be loaded is stored locally or not based on the identifier of the mirror image to be loaded; and responding to the local storage of the mirror image to be loaded, and sending the mirror image to be loaded to the computing node.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or computer device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection according to one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method of any of the method embodiments described above may be implemented, and an execution manner and beneficial effects of the method are similar, and are not described herein again.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is merely exemplary of the present application and is presented to enable those skilled in the art to understand and practice the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A circuit board grabbing control method is used for controlling a mechanical arm clamping jaw to grab a circuit board on a belt transmission line; characterized in that the method comprises:
acquiring shot images formed by shooting the circuit board at two moments;
calculating pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments;
calculating the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments;
judging whether the pose deviation is smaller than a preset deviation or not;
and controlling a mechanical arm clamping jaw to grab the circuit board in response to the pose deviation being smaller than the preset deviation.
2. The method of claim 1, wherein the captured image comprises a depth image;
the calculating of the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments comprises:
generating a corresponding source point cloud based on the depth images shot at two moments;
acquiring registration point clouds which are registered with target point clouds in the source point clouds corresponding to the two moments, wherein the target point clouds are acquired in advance and represent point clouds of similar circuit board outlines;
and calculating pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
3. The method of claim 2, wherein the captured image further comprises a planar image;
the calculating the pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments further comprises:
acquiring sub-image areas matched with a target plane image in the plane images corresponding to two moments, wherein the target plane image is a plane image which is acquired in advance and represents the appearance of the same type of circuit board;
determining a corresponding sub-depth image in the depth image based on the sub-image regions corresponding to the two moments;
generating a corresponding source point cloud based on the depth images corresponding to the two moments, including: and generating the corresponding source point cloud based on the sub-depth images in the depth images corresponding to the two moments.
4. The method according to claim 2 or 3,
the calculating of the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments comprises the following steps:
determining position coordinates of a set position in the circuit board at two moments based on the registration point clouds corresponding to the two moments;
the calculating of the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments comprises:
and calculating the position deviation based on the position coordinates of the preset position in the circuit board at two moments.
5. The method according to claim 2 or 3,
the calculating of the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments comprises the following steps:
respectively calculating the principal component directions of the minimum directed bounding boxes which surround the circuit board at two moments based on the registration point clouds corresponding to the two moments;
the calculating of the pose deviation of the circuit board at two moments based on the pose parameters of the circuit board at two moments comprises:
and calculating the angle deviation of the circuit board at two moments based on the principal component directions of the minimum directed bounding boxes corresponding to the two moments.
6. The method of claim 1, wherein controlling the robot arm gripper to grip the circuit board comprises:
acquiring grabbing offset parameters of the circuit board;
calculating the grabbing pose of the mechanical arm when the mechanical arm grabs the circuit board based on the grabbing bias parameters of the circuit board and the pose parameters of the circuit board at the later moment of the two moments;
and controlling the mechanical arm clamping jaw to clamp the circuit board based on the clamping pose.
7. A circuit board grabbing control device is used for controlling a mechanical arm clamping jaw to grab a circuit board on a belt transmission line; it is characterized by comprising:
the image acquisition unit is used for acquiring shot images formed by the shooting circuit boards at two moments;
the pose parameter determining unit is used for calculating pose parameters of the circuit board at two moments based on the shot images corresponding to the two moments;
the pose deviation determining unit is used for calculating pose deviations of the circuit board at two moments based on pose parameters of the circuit board at two moments;
the judging unit is used for judging whether the pose deviation is within a preset deviation or not;
and the grabbing control unit is used for controlling the clamping jaw of the mechanical arm to grab the circuit board under the condition that the judgment unit judges that the pose deviation is within the preset deviation.
8. The apparatus of claim 7, wherein the captured image comprises a depth image; the pose parameter determination unit includes:
a source point cloud determination subunit, configured to generate a corresponding source point cloud based on the depth images captured at two moments;
the registration point cloud determining subunit is used for acquiring registration point clouds which are registered with target point clouds in the source point clouds corresponding to the two moments, wherein the target point clouds are acquired in advance and represent point clouds of similar circuit board outlines;
and the pose parameter determining subunit is used for calculating the pose parameters of the circuit board at two moments based on the registration point clouds corresponding to the two moments.
9. A computing device, comprising: a memory and a processor, wherein the memory has stored therein a computer program which, when executed by the processor, implements the circuit board capture control method of any of claims 1-6.
10. A computer-readable storage medium, wherein the storage medium stores computer instructions for causing the computer to execute the circuit board capture control method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111331010.XA CN113752268B (en) | 2021-11-11 | 2021-11-11 | Circuit board grabbing control method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111331010.XA CN113752268B (en) | 2021-11-11 | 2021-11-11 | Circuit board grabbing control method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113752268A true CN113752268A (en) | 2021-12-07 |
| CN113752268B CN113752268B (en) | 2022-01-04 |
Family
ID=78784889
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111331010.XA Active CN113752268B (en) | 2021-11-11 | 2021-11-11 | Circuit board grabbing control method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113752268B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1806940A (en) * | 2006-01-23 | 2006-07-26 | 湖南大学 | Defective goods automatic sorting method and equipment for high-speed automated production line |
| CN102177003A (en) * | 2008-10-07 | 2011-09-07 | 川崎重工业株式会社 | Substrate transfer robot and system |
| CN105817422A (en) * | 2016-04-21 | 2016-08-03 | 石河子大学 | Robot-based automatic-collection random-placing express package information system and method thereof |
| JP2017109820A (en) * | 2015-12-15 | 2017-06-22 | 澁谷工業株式会社 | Article processing device |
| CN110315525A (en) * | 2018-03-29 | 2019-10-11 | 天津工业大学 | A kind of robot workpiece grabbing method of view-based access control model guidance |
| CN111347423A (en) * | 2020-01-19 | 2020-06-30 | 天津大学 | Dynamic tracking gluing method for industrial robot conveyor belt |
| CN111483803A (en) * | 2020-04-17 | 2020-08-04 | 湖南视比特机器人有限公司 | Control method, capture system and storage medium |
| CN111591667A (en) * | 2020-06-11 | 2020-08-28 | 广州市泰立机电设备有限公司 | Circuit board collecting machine |
| CN213001307U (en) * | 2020-07-10 | 2021-04-20 | 季华实验室 | Sorting device based on vision system and millimeter wave sensor |
-
2021
- 2021-11-11 CN CN202111331010.XA patent/CN113752268B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1806940A (en) * | 2006-01-23 | 2006-07-26 | 湖南大学 | Defective goods automatic sorting method and equipment for high-speed automated production line |
| CN102177003A (en) * | 2008-10-07 | 2011-09-07 | 川崎重工业株式会社 | Substrate transfer robot and system |
| JP2017109820A (en) * | 2015-12-15 | 2017-06-22 | 澁谷工業株式会社 | Article processing device |
| CN105817422A (en) * | 2016-04-21 | 2016-08-03 | 石河子大学 | Robot-based automatic-collection random-placing express package information system and method thereof |
| CN110315525A (en) * | 2018-03-29 | 2019-10-11 | 天津工业大学 | A kind of robot workpiece grabbing method of view-based access control model guidance |
| CN111347423A (en) * | 2020-01-19 | 2020-06-30 | 天津大学 | Dynamic tracking gluing method for industrial robot conveyor belt |
| CN111483803A (en) * | 2020-04-17 | 2020-08-04 | 湖南视比特机器人有限公司 | Control method, capture system and storage medium |
| CN111591667A (en) * | 2020-06-11 | 2020-08-28 | 广州市泰立机电设备有限公司 | Circuit board collecting machine |
| CN213001307U (en) * | 2020-07-10 | 2021-04-20 | 季华实验室 | Sorting device based on vision system and millimeter wave sensor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113752268B (en) | 2022-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9652660B2 (en) | Control system, robot system, and control method | |
| JP4794011B2 (en) | Image processing apparatus and robot control system | |
| US20220414910A1 (en) | Scene contour recognition method and apparatus, computer-readable medium, and electronic device | |
| JPWO2020138345A1 (en) | Article position management device, article position management system, article position management method and program | |
| CN115047890B (en) | Unmanned ship control method, unmanned ship control device and computer-readable storage medium | |
| CN113752268B (en) | Circuit board grabbing control method, device, equipment and storage medium | |
| JP6772630B2 (en) | 3D measuring device and 3D object recognition method | |
| JP7250489B2 (en) | Image processing device, its control method, and program | |
| JPH03228589A (en) | Positioning method for work | |
| CN115482286A (en) | Calibration verification method, device, equipment and medium for camera internal parameters | |
| WO2023213252A1 (en) | Scanning data processing method and apparatus, and device and medium | |
| CN116182878B (en) | Road curved surface information generation method, device, equipment and computer readable medium | |
| JP5514062B2 (en) | Electronic device, imaging screen display method with information, and program | |
| WO2022194158A1 (en) | Target tracking method and apparatus, device, and medium | |
| CN115086541B (en) | Shooting position determining method, device, equipment and medium | |
| CN114140538A (en) | Vehicle-mounted camera pose adjusting method, device, equipment and computer readable medium | |
| CN111083310B (en) | Data synchronization processing method and system for 2D linear array camera and 3D linear array camera | |
| CN114022342A (en) | Acquisition method and device for acquisition point information, electronic equipment and storage medium | |
| CN114022341A (en) | Acquisition method and device for acquisition point information, electronic equipment and storage medium | |
| Varga et al. | Experimental vision system setup based on the serial configuration interface | |
| US10078880B2 (en) | Image processing device and image processing system | |
| CN112880675B (en) | Pose smoothing method and device for visual positioning, terminal and mobile robot | |
| CN113023353B (en) | Automatic clamping, loading and unloading carrying method and device based on image recognition and positioning technology | |
| CN116258811B (en) | Information transmission method, apparatus, electronic device, and computer-readable medium | |
| Vagas | Approaches for the 3d camera system image processing at automated workplaces |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |