CN115564847B - Visual calibration method and device of visual assembly system and storage medium - Google Patents
Visual calibration method and device of visual assembly system and storage medium Download PDFInfo
- Publication number
- CN115564847B CN115564847B CN202211438841.1A CN202211438841A CN115564847B CN 115564847 B CN115564847 B CN 115564847B CN 202211438841 A CN202211438841 A CN 202211438841A CN 115564847 B CN115564847 B CN 115564847B
- Authority
- CN
- China
- Prior art keywords
- moving mechanism
- image
- coordinate system
- controlling
- image coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The disclosure provides a vision calibration method and device of a vision assembly system and a storage medium. The method comprises the following steps: calibrating the incidence relation between the first image coordinate system and the second image coordinate system; calibrating the conversion relation between the space coordinate of the first moving mechanism and the coordinate of the first moving mechanism in a first image coordinate system; calibrating the conversion relation between the space coordinate of the second moving mechanism and the coordinate of the second moving mechanism in a second image coordinate system; wherein the first image coordinate system is an image coordinate system of the lower camera and the second image coordinate system is an image coordinate system of the upper camera.
Description
Technical Field
The present disclosure relates to machine vision technology, and more particularly, to a method and apparatus for calibrating vision of a vision assembly system, and a storage medium.
Background
The traditional up-down visual calibration method is roughly divided into two methods, wherein one camera is used for nine-point calibration, the other camera is associated with the camera, and the other method is used for associating the two cameras with a mechanical axis respectively, and the two methods require that two axes can be seen in the visual field of at least one camera to operate.
In some cases, due to the product and mechanism design, only one axis of motion can be seen in the visual fields of the upper camera and the lower camera respectively, and the traditional calibration mode cannot be used. Therefore, it is necessary to provide a visual calibration scheme suitable for such a situation.
Disclosure of Invention
An object of the present disclosure is to provide a method and an apparatus for calibrating vision of a vision assembling system, and a storage medium, which can be applied to a case where only one axis of motion is visible in the upper and lower camera views, respectively.
According to a first aspect of the present disclosure, a vision calibration method of a vision mounting system is provided. The vision assembly system comprises a first rail, a second rail, a first moving mechanism movable on the first rail, and a second moving mechanism movable on the second rail; a clamping device is fixed on the first moving mechanism, a tool is fixed on the second moving mechanism, and the projections of the first rail and the second rail on the horizontal plane form an included angle; the vision assembly system further comprises a lower camera and an upper camera; the lower camera is fixedly arranged at a first position in space and is used for observing a first moving mechanism which moves to a first observation interval from the lower part; the upper camera is fixedly arranged at a second position in the space and is used for observing the second moving mechanism which moves to the second observation interval from the upper part. The method comprises the following steps: calibrating the incidence relation between the first image coordinate system and the second image coordinate system; calibrating the conversion relation between the space coordinate of the first moving mechanism and the coordinate of the first moving mechanism in a first image coordinate system; calibrating the conversion relation between the space coordinate of the second moving mechanism and the coordinate of the second moving mechanism in a second image coordinate system; wherein the first image coordinate system is an image coordinate system of the lower camera and the second image coordinate system is an image coordinate system of the upper camera.
According to a second aspect of the present disclosure, a visual calibration apparatus is provided. The vision calibration apparatus comprises a memory for storing computer instructions and a processor for retrieving the computer instructions from the memory to perform the method according to any one of the first aspect of the present disclosure.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, implement the method according to any one of the first aspects of the present disclosure.
The vision calibration method of the vision assembly system comprises a first rail, a second rail, a first moving mechanism capable of moving on the first rail and a second moving mechanism capable of moving on the second rail, wherein a clamping device is fixed on the first moving mechanism, a tool is fixed on the second moving mechanism, and the projections of the first rail and the second rail on a horizontal plane form an included angle; the lower camera is fixedly arranged at a first position in space and is used for observing the first moving mechanism which moves to the first observation interval from the lower part; the upper camera is fixedly arranged at a second position in the space and is used for observing the second moving mechanism which moves to the second observation interval from the upper part. In the visual assembly system with the structure, the lower camera can only observe part of the first track but not the second track, and the upper camera can only observe part of the second track but not the first track.
Other features of the present disclosure and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a schematic view of a vision assembly system of an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart diagram of a vision calibration method of the vision assembly system of an embodiment of the present disclosure;
FIG. 3 is a schematic view of a calibration object of one embodiment of the present disclosure;
FIG. 4 is a schematic view of a calibration object of another embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a visual calibration apparatus according to an embodiment of the disclosure;
fig. 6 is a hardware configuration diagram of a visual calibration apparatus according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
The disclosed embodiments provide a vision mounting system, fig. 1 is a top view of the vision mounting system.
The vision assembly system includes a first rail, a second rail, a first movement mechanism (not shown in fig. 1) movable on the first rail, and a second movement mechanism (not shown in fig. 1) movable on the second rail. The first moving mechanism is fixed with a clamping device, and the second moving mechanism is fixed with a tool. The first rail and the second rail are in a separated relation, the movement of the first moving mechanism on the first rail and the movement of the second moving mechanism on the second rail are independent of each other, and the projections of the first rail and the second rail on a horizontal plane form an included angle.
The vision assembly system also includes a lower camera and an upper camera. The lower camera is fixedly arranged at a first position in the space and is used for observing the first moving mechanism which moves to the first observation interval from the lower part. The upper camera is fixedly arranged at a second position in the space and is used for observing the second moving mechanism which moves to the second observation interval from the upper part.
In the visual assembly system with the structure, the lower camera can only observe part of the first track but not the second track, and the upper camera can only observe part of the second track but not the first track, so that the traditional up-and-down visual calibration method is not suitable for the scene.
In some embodiments, the grasping device is a gripper or a suction nozzle. The clamping device is used for fixing the material A. The tool is used for placing the semi-finished product B. When the first moving mechanism and the second moving mechanism move to enable the material A on the clamping device to be aligned with the semi-finished product B on the tool, the material A and the semi-finished product B can be assembled. Therefore, the vision guiding device needs to be accurately calibrated to ensure that the material A on the clamping device and the semi-finished product B on the tool can be accurately aligned and meet the requirement of assembly on clearance.
In some embodiments, the first track is arranged parallel to an X-axis and the second track is arranged parallel to a Y-axis, the X-axis and the Y-axis being parallel to a horizontal plane, the X-axis and the Y-axis having an included angle. In some embodiments, the X-axis and the Y-axis are at a 90 degree angle.
Referring to fig. 2, an embodiment of the present disclosure provides a visual calibration method of an assembly system, which may be used for visual calibration of the assembly system shown in fig. 1. The vision calibration method comprises steps S101-S103.
S101, calibrating the incidence relation between the first image coordinate system and the second image coordinate system.
S102, calibrating the conversion relation between the space coordinate of the first moving mechanism and the coordinate of the first moving mechanism in the first image coordinate system.
S103, calibrating the conversion relation between the space coordinate of the second moving mechanism and the coordinate of the second moving mechanism in the second image coordinate system.
The first image coordinate system is an image coordinate system of the lower camera, and the second image coordinate system is an image coordinate system of the upper camera. The first image coordinate system and the second image coordinate system are two-dimensional coordinate systems, which may be pixel coordinate systems.
In the disclosed embodiment, the spatial coordinate system is a coordinate system constructed based on an X-axis and a Y-axis, and the spatial coordinates include an X-axis coordinate and a Y-axis coordinate.
According to the visual calibration method of the assembly system, the association relationship between a first image coordinate system and a second image coordinate system is calibrated respectively, the conversion relationship between the space coordinate of a first moving mechanism and the coordinate of the first moving mechanism in the first image coordinate system is calibrated, the conversion relationship between the space coordinate of a second moving mechanism and the coordinate of the second moving mechanism in the second image coordinate system is calibrated, through the three calibration steps, a lower camera is associated with a first track and a second track respectively, an upper camera is associated with the first track and the second track respectively, and the visual calibration of the visual assembly system with separated upper and lower camera fields of vision is realized.
In some embodiments, calibrating the association relationship between the first image coordinate system and the second image coordinate system includes steps S1011 to S1015:
s1011, controlling the clamping device to clamp the calibration object, wherein at least three mark points positioned on different straight lines are arranged on the calibration object.
The marker can be placed in a semi-finished product B on the tool and meets the requirement of assembly precision clearance. In some embodiments, the shape of the marker is the same as the shape of material a, or the sample material is used directly, i.e. material a is used as the marker. Referring to fig. 3, the center of the marker is provided with a notch, and several edge points of the notch can be used as the marker points. Referring to FIG. 4, the markers have marker points 1-5 thereon.
S1012, controlling the first moving mechanism to move to the first photographing position in the first observation interval, and controlling the lower camera to photograph to obtain a first photographing position image.
And S1013, controlling the first moving mechanism and the second moving mechanism to move to the assembling position, and assembling the calibration object on the semi-finished product on the tool.
And S1014, controlling the second moving mechanism to move to the second photographing position in the second observation interval, and controlling the upper camera to photograph to obtain a second photographing position image.
And S1015, determining the association relationship between the first image coordinate system and the second image coordinate system according to the position of the mark point in the first photographing position image in the first image coordinate system and the position of the mark point in the second image in the second photographing position image coordinate system.
In the embodiment of the present disclosure, controlling the first moving mechanism and the second moving mechanism to move to the assembling position means: the first moving mechanism is controlled to move on the first track, the second moving mechanism is controlled to move on the second track, the material A or the calibration object on the first moving mechanism is aligned with the semi-finished product B on the second moving mechanism, the assembling precision requirement is met, and assembling can be carried out.
The first photographing position is a photographing position of the first moving mechanism in an actual assembly process of the visual guidance assembly system, that is, when the first moving mechanism moves to the first photographing position, the camera is controlled to photograph, and the target position of the first moving mechanism is determined according to an image obtained by photographing at the moment and the calibration result of the embodiment. The second photographing position is a photographing position of the second moving mechanism in an actual assembly process of the vision guiding assembly system, that is, when the second moving mechanism moves to the second photographing position, the upper camera is controlled to photograph, and the target position of the second moving mechanism is determined according to an image obtained by photographing at the moment and the calibration result of the embodiment. When the first moving mechanism and the second moving mechanism move to the respective target positions, the material A on the first moving mechanism and the semi-finished product B on the second moving mechanism are aligned and meet the assembly precision requirement, and assembly can be carried out.
In some embodiments, the association relationship between the first image coordinate system and the second image coordinate system is determined according to the positions of the plurality of marker points in the first image coordinate system and the positions of the plurality of marker points in the second image coordinate system. For example, the association relationship between the first image coordinate system and the second image coordinate system may be determined based on the one-to-one correspondence relationship of the coordinates in the first image coordinate system of the 5 marker points in the first photographed image and the coordinates in the second image coordinate system of the 5 marker points in the second photographed image.
In some embodiments, the calibrated association between the first image coordinate system and the second image coordinate system may include a translation coefficient, a scaling coefficient, a rotation coefficient, and a tilt coefficient therebetween.
In some embodiments, calibrating the transformation relationship between the spatial coordinates of the first moving mechanism and the coordinates of the first moving mechanism in the first image coordinate system includes steps S1021-S1025:
and S1021, controlling the clamping device to clamp the calibration object.
S1022, the first moving mechanism is controlled to move to the first position in the first observation interval, and the camera is controlled to take a picture to obtain the first position image.
And S1023, controlling the first moving mechanism to move to a second position in the first observation interval, and controlling the camera to shoot to obtain a second position image.
And S1024, controlling the first moving mechanism to move to a third position in the first observation interval, and controlling the lower camera to shoot to obtain a third position image.
And S1025, determining the conversion relation between the space coordinates of the first moving mechanism and the coordinates of the first moving mechanism in the first image coordinate system according to the space coordinates of the first position, the second position and the third position and the image coordinates of the calibration object in the first position image, the second position image and the third position image.
Referring to fig. 1, the first position is P11, the second position is P12, and the third position is P13. In some embodiments, the second position is a first photographing position, the first position and the third position are respectively located at two sides of the second position, and the distances from the first position to the second position are the same as the distances from the third position to the second position. The spatial coordinate of the first position is (x) 12 -∆x，y 12 ) The spatial coordinate of the second position is (x) 12 ，y 12 ) Space of the third positionThe coordinate is (x) 12 +∆x，y 12 ). The image coordinate of the calibration object in the first position image is (u) 11 ,v 11 ) And the image coordinate of the calibration object in the second position image is (u) 12 ,v 12 ) And the image coordinate of the calibration object in the third position image is (u) 13 ,v 13 ). According to the one-to-one correspondence relationship among the spatial coordinates of the first position, the second position and the third position and the image coordinates of the calibration object in the first position image, the image coordinates of the calibration object in the second position image and the image coordinates of the calibration object in the third position image, the conversion relationship between the spatial coordinates of the first moving mechanism and the coordinates of the first moving mechanism in the first image coordinate system can be determined.
In some embodiments, the X-axis coordinate of the spatial coordinates of the first, second, and third positions may be read directly from the controller associated with the first movement mechanism, and the Y-axis coordinate may be the same predetermined value.
In some embodiments, the calibrated transformation relationship between the mechanical axis coordinates of the first moving mechanism and the coordinates of the first moving mechanism in the first image coordinate system may include a translation coefficient, a scaling coefficient, a rotation coefficient, and a tilt coefficient therebetween.
In some embodiments, the translation relationship between the spatial coordinates of the second moving mechanism and the coordinates of the second moving mechanism in the second image coordinate system is calibrated, including steps S1031 to S1035.
And S1031, controlling the first moving mechanism and the second moving mechanism to move to the assembling position, and assembling the calibration object clamped on the clamping device on the semi-finished product on the tool.
And S1032, controlling the second moving mechanism to move to a fourth position in the second observation interval, and controlling the upper camera to take a picture to obtain a fourth position image.
And S1033, controlling the second moving mechanism to move to a fifth position in the second observation interval, and controlling the upper camera to take a picture to obtain a fifth position image.
S1034, controlling the second moving mechanism to move to a sixth position in the second observation interval, and controlling the camera to shoot to obtain a sixth position image.
S1035, determining a conversion relationship between the spatial coordinates of the second moving mechanism and the coordinates of the second moving mechanism in the second image coordinate system, according to the spatial coordinates of the fourth position, the fifth position, and the sixth position, and the image coordinates of the calibration object in the fourth position image, the fifth position image, and the sixth position image.
Referring to fig. 1, the fourth position is P21, the fifth position is P22, and the sixth position is P23. In some embodiments, the fifth position is a second photographing position, the fourth position and the sixth position are respectively located at two sides of the fifth position, and the distances from the fourth position to the fifth position are the same. The spatial coordinate of the fourth position is (x) 22 ，y 22 Δ y) the space coordinate of the fifth position is (x) 22 ，y 22 ) The spatial coordinate of the sixth position is (x) 22 ，y 22 Δ y). The image coordinate of the calibration object in the first position image is (u) 21 ,v 21 ) And the image coordinate of the calibration object in the second position image is (u) 22 ,v 22 ) And the image coordinate of the calibration object in the third position image is (u) 23 ,v 23 ). According to the one-to-one correspondence relationship among the spatial coordinates of the fourth position, the fifth position and the sixth position, the image coordinates of the calibration object in the fourth position image, the image coordinates of the calibration object in the fifth position image, and the image coordinates of the calibration object in the sixth position image, the conversion relationship between the spatial coordinates of the second moving mechanism and the coordinates of the second moving mechanism in the second image coordinate system can be determined.
In some embodiments, the Y-axis coordinate of the spatial coordinates of the fourth, fifth, and sixth positions may be read directly from the controller associated with the second moving mechanism, and the X-axis coordinate may be the same predetermined value.
In some embodiments, the calibrated conversion relationship between the mechanical axis coordinates of the second moving mechanism and the coordinates of the second moving mechanism in the second image coordinate system may include a translation coefficient, a scaling coefficient, a rotation coefficient, and a tilt coefficient therebetween.
The vision calibration method of the vision assembly system comprises a first rail, a second rail, a first moving mechanism capable of moving on the first rail and a second moving mechanism capable of moving on the second rail, wherein a clamping device is fixed on the first moving mechanism, a tool is fixed on the second moving mechanism, and the projections of the first rail and the second rail on a horizontal plane form an included angle; the lower camera is fixedly arranged at a first position in space and is used for observing the first moving mechanism which moves to the first observation interval from the lower part; the upper camera is fixedly arranged at a second position in the space and is used for observing the second moving mechanism which moves to the second observation interval from the upper part. In the visual assembling system with the structure, the lower camera can only observe part of the first track but not the second track, and the upper camera can only observe part of the second track but not the first track.
Referring to fig. 5, an embodiment of the present disclosure provides a visual calibration apparatus 400. The vision calibration apparatus 400 includes a processor 410 and a memory 420. The memory 420 is used for storing computer instructions, and the processor 410 is used for calling the computer instructions from the memory 420 to execute the visual calibration method in any embodiment of the present disclosure.
In some embodiments, the visual calibration apparatus may be a computer device. For example, the visual calibration apparatus may be a terminal or a server. Furthermore, the terminal can be a portable computer, a tablet computer, a palm computer, an intelligent control platform and the like. The server can be a cloud server and the like.
For example, referring to fig. 6, the visual targeting device may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a central processing unit CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, a ROM (read only memory), a RAM (random access memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. Communication device 1400 is capable of wired or wireless communication, for example. The display device 1500 is, for example, a liquid crystal display panel, a touch panel, or the like. The input device 1600 may include, for example, a touch screen, a keyboard, and the like. A user can input/output voice information through the speaker 1700 and the microphone 1800.
Although a plurality of devices are shown for the visual calibration device in fig. 6, the present disclosure may refer to only some of the devices, for example, the visual calibration device refers only to the memory 1200 and the processor 1100.
In the embodiment of the present disclosure, the memory 1200 of the vision calibration apparatus is used to store instructions for controlling the processor 1100 to execute the vision calibration method provided by the embodiment of the present disclosure.
In the above description, the skilled person can design the instructions according to the disclosed solution of the present disclosure. How the instructions control the operation of the processor is well known in the art and will not be described in detail herein.
Embodiments of the present disclosure provide a computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor, implement a vision calibration method in any embodiment of the present disclosure.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as a punch card or an in-groove protruding structure with instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the disclosure are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims (10)
1. The visual calibration method of the visual assembly system is characterized in that the visual assembly system comprises a first rail, a second rail, a first moving mechanism capable of moving on the first rail and a second moving mechanism capable of moving on the second rail; the first moving mechanism is fixed with a clamping device, the second moving mechanism is fixed with a tool, the first rail and the second rail are in a separated relation, and the projections of the first rail and the second rail on the horizontal plane form an included angle; the vision assembly system further comprises a lower camera and an upper camera; the lower camera is fixedly arranged at a first position in space and is used for observing a first moving mechanism which moves to a first observation interval from the lower part; the upper camera is fixedly arranged at a second position in the space and is used for observing a second moving mechanism which moves to a second observation interval from the upper part;
the method comprises the following steps:
calibrating the incidence relation between the first image coordinate system and the second image coordinate system;
calibrating the conversion relation between the space coordinate of the first moving mechanism and the coordinate of the first moving mechanism in a first image coordinate system;
calibrating the conversion relation between the space coordinate of the second moving mechanism and the coordinate of the second moving mechanism in a second image coordinate system;
wherein the first image coordinate system is an image coordinate system of the lower camera and the second image coordinate system is an image coordinate system of the upper camera.
2. The method of claim 1, wherein calibrating the relationship between the first image coordinate system and the second image coordinate system comprises:
controlling the clamping device to clamp a calibration object, wherein at least three mark points positioned on different straight lines are arranged on the calibration object;
controlling the first moving mechanism to move to a first photographing position in the first observation interval, and controlling the lower camera to photograph to obtain a first photographing position image;
controlling the first moving mechanism and the second moving mechanism to move to an assembling position, and assembling the calibration object on the semi-finished product on the tool;
controlling the second moving mechanism to move to a second photographing position in the second observation interval, and controlling the upper camera to photograph to obtain a second photographing position image;
and determining the association relationship between the first image coordinate system and the second image coordinate system according to the positions of the mark points in the first photographing position image in the first image coordinate system and the positions of the mark points in the second photographing position image in the second image coordinate system.
3. The method of claim 1, wherein calibrating the transformation relationship between the spatial coordinates of the first movement mechanism and the coordinates of the first movement mechanism in the first image coordinate system comprises:
controlling the clamping device to clamp the calibration object;
controlling the first moving mechanism to move to a first position in the first observation interval, and controlling the lower camera to take a picture to obtain a first position image;
controlling the first moving mechanism to move to a second position in the first observation interval, and controlling the lower camera to take a picture to obtain a second position image;
controlling the first moving mechanism to move to a third position in the first observation interval, and controlling the lower camera to take a picture to obtain a third position image;
and determining the conversion relation between the space coordinates of the first moving mechanism and the coordinates of the first moving mechanism in a first image coordinate system according to the space coordinates of the first position, the second position and the third position and the image coordinates of the calibration objects in the first position image, the second position image and the third position image.
4. The method of claim 3, wherein the second position is a first photographing position, and the first position and the third position are respectively located at two sides of the second position.
5. The method according to claim 1, wherein calibrating the transformation relationship between the spatial coordinates of the second moving mechanism and the coordinates of the second moving mechanism in the second image coordinate system comprises:
controlling the first moving mechanism and the second moving mechanism to move to an assembling position, and assembling the calibration object clamped on the clamping device on the semi-finished product on the tooling;
controlling the second moving mechanism to move to a fourth position in the second observation interval, and controlling the upper camera to take a picture to obtain a fourth position image;
controlling the second moving mechanism to move to a fifth position in the second observation interval, and controlling the upper camera to take a picture to obtain a fifth position image;
controlling the second moving mechanism to move to a sixth position in the second observation interval, and controlling the upper camera to take a picture to obtain a sixth position image;
and determining the conversion relation between the space coordinates of the second moving mechanism and the coordinates of the second moving mechanism in a second image coordinate system according to the space coordinates of the fourth position, the fifth position and the sixth position and the image coordinates of the calibration objects in the fourth position image, the fifth position image and the sixth position image.
6. The method of claim 5, wherein the fifth position is a second photographing position, and the fourth position and the sixth position are respectively located at two sides of the fifth position.
7. The method of any one of claims 2 to 6, wherein the calibration object is a sample material.
8. A method according to any one of claims 1-6, wherein the gripping means is a gripper or a suction nozzle.
9. A vision calibration arrangement, comprising a memory for storing computer instructions and a processor for retrieving said computer instructions from said memory for performing the method of any one of claims 1-8.
10. A computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, implement the method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438841.1A CN115564847B (en) | 2022-11-17 | 2022-11-17 | Visual calibration method and device of visual assembly system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211438841.1A CN115564847B (en) | 2022-11-17 | 2022-11-17 | Visual calibration method and device of visual assembly system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115564847A CN115564847A (en) | 2023-01-03 |
| CN115564847B true CN115564847B (en) | 2023-03-31 |
Family
ID=84769825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211438841.1A Active CN115564847B (en) | 2022-11-17 | 2022-11-17 | Visual calibration method and device of visual assembly system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115564847B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022120567A1 (en) * | 2020-12-08 | 2022-06-16 | 深圳先进技术研究院 | Automatic calibration system based on visual guidance |
| CN112894823B (en) * | 2021-02-08 | 2022-06-21 | 珞石（山东）智能科技有限公司 | Robot high-precision assembling method based on visual servo |
| CN112991461A (en) * | 2021-03-11 | 2021-06-18 | 珠海格力智能装备有限公司 | Material assembling method and device, computer readable storage medium and processor |
| CN113997059A (en) * | 2021-11-02 | 2022-02-01 | 珠海格力智能装备有限公司 | Compressor workpiece assembling method, device and system and storage medium |
| CN114332247A (en) * | 2022-01-07 | 2022-04-12 | 湖南视比特机器人有限公司 | Calibration method and device for multi-view vision measurement, storage medium and camera equipment |
-
2022
- 2022-11-17 CN CN202211438841.1A patent/CN115564847B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115564847A (en) | 2023-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107491174B (en) | Method, device and system for remote assistance and electronic equipment | |
| CN113238650B (en) | Gesture recognition and control method and device and virtual reality equipment | |
| EP4198694A1 (en) | Positioning and tracking method and platform, head-mounted display system, and computer-readable storage medium | |
| CN110503689B (en) | Pose prediction method, model training method and model training device | |
| US20060284791A1 (en) | Augmented reality system and method with mobile and interactive function for multiple users | |
| CN112115799B (en) | Three-dimensional gesture recognition method, device and equipment based on marked points | |
| CN111766951B (en) | Image display method and apparatus, computer system, and computer-readable storage medium | |
| US11675178B2 (en) | Virtual slide stage (VSS) method for viewing whole slide images | |
| US10572127B2 (en) | Display control of an image on a display screen | |
| CN111462238A (en) | Attitude estimation optimization method and device and storage medium | |
| CN113255619A (en) | Lane line recognition and positioning method, electronic device, and computer-readable medium | |
| CN113997295A (en) | Hand-eye calibration method and device for mechanical arm, electronic equipment and storage medium | |
| CN115617166A (en) | Interaction control method and device and electronic equipment | |
| CN115564847B (en) | Visual calibration method and device of visual assembly system and storage medium | |
| WO2024066756A1 (en) | Interaction method and apparatus, and display device | |
| CN110837766B (en) | Gesture recognition method, gesture processing method and device | |
| CN116244024A (en) | Interactive control method and device, head-mounted display equipment and medium | |
| CN111429519B (en) | Three-dimensional scene display method and device, readable storage medium and electronic equipment | |
| CN115576457A (en) | Display control method and device, augmented reality head-mounted device and medium | |
| CN112883754B (en) | Auxiliary operation system of forklift | |
| CN115396741A (en) | Panoramic video playing method and device, electronic equipment and readable storage medium | |
| CN111784772B (en) | Attitude estimation model training method and device based on domain randomization | |
| CN110322484B (en) | Calibration method and system for multi-device shared augmented reality virtual space | |
| CN113730905A (en) | Method and device for realizing free migration in virtual space | |
| US10281294B2 (en) | Navigation system and navigation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |