CN114670194B - Positioning method and device for manipulator system - Google Patents
Positioning method and device for manipulator system Download PDFInfo
- Publication number
- CN114670194B CN114670194B CN202210286905.4A CN202210286905A CN114670194B CN 114670194 B CN114670194 B CN 114670194B CN 202210286905 A CN202210286905 A CN 202210286905A CN 114670194 B CN114670194 B CN 114670194B
- Authority
- CN
- China
- Prior art keywords
- equipment
- coordinates
- coordinate system
- camera
- mechanical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/1635—Programme controls characterised by the control loop flexible-arm control
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
- B25J19/02—Sensing devices
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1661—Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1692—Calibration of manipulator
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Manipulator (AREA)
Abstract
The application provides a mechanical arm system positioning method and a mechanical arm system positioning device, wherein the method obtains a first mapping relation between a mechanical coordinate system and a camera coordinate system; acquiring a second mapping relation between a camera coordinate system and a device coordinate system of the electronic device; obtaining a third mapping relation between the equipment coordinate system and the mechanical coordinate system based on the first mapping relation and the second mapping relation; and acquiring the equipment coordinates of the target position of the electronic equipment in the equipment coordinate system, and converting the equipment coordinates to obtain the mechanical coordinates of the target position based on the third mapping relation. According to the scheme, the mechanical position of the target position is not required to be obtained according to the mechanical arm equipment, the development and debugging process of the automatic script can be decoupled from the mechanical arm equipment, and the development and debugging efficiency of the automatic script is improved. Meanwhile, the mechanical coordinates corresponding to the target position do not need to be obtained by a camera, so that the time consumption of the mechanical coordinate obtaining process of the target position is saved, and the testing efficiency of the whole testing process is improved.
Description
Technical Field
The application relates to the technical field of machine vision systems, in particular to a positioning method and device for a manipulator system.
Background
The automatic test, automatic assembly and the like of the equipment can be realized by using the manipulator system, and the automatic test equipment is widely applied to industrial production. The machine vision system is an important component of the manipulator system, and corresponds to eyes of the manipulator system, and position coordinates of a tested item on the equipment are obtained through the machine vision system. Further, the manipulator device is controlled to move to the position coordinates to execute corresponding operations, such as sliding, clicking, long-pressing, and the like, on a screen of the electronic device, so as to realize automatic testing, such as pressure testing and performance testing, of the target electronic device. But current robotic systems are inefficient.
Disclosure of Invention
In view of this, the present application provides a method and an apparatus for positioning a manipulator system, so as to solve the above technical problems, and the disclosed technical solution is as follows:
in a first aspect, the present application provides a method for positioning a manipulator system, the method comprising: acquiring a first mapping relation between a mechanical coordinate system and a camera coordinate system; acquiring a second mapping relation between a camera coordinate system and a device coordinate system of the electronic device; obtaining a third mapping relation between the equipment coordinate system and the mechanical coordinate system based on the first mapping relation and the second mapping relation; and acquiring the equipment coordinates of the target position of the electronic equipment in the equipment coordinate system, and converting the equipment coordinates to obtain the mechanical coordinates of the target position based on the third mapping relation. Therefore, the mechanical arm system positioning method provided by the scheme can directly convert the equipment coordinates corresponding to the target position of the electronic equipment into mechanical coordinates, and further control the mechanical arm equipment to move to the target position based on the mechanical coordinates and execute corresponding operations. According to the scheme, the equipment coordinates of the target position can be obtained through the tool, or the equipment coordinates of the target position are directly written into the automation script, so that the mechanical position of the target position is not required to be written into according to the mechanical hand equipment, the development and debugging process of the automation script can be decoupled from the mechanical hand equipment, and the development and debugging efficiency of the automation script is improved. Meanwhile, the camera coordinate of the target position is not required to be obtained by the camera by utilizing the scheme, and then the camera coordinate is converted into the mechanical coordinate, so that the time consumption of the process of obtaining the mechanical coordinate of the target position is saved, and the testing efficiency of the whole testing process is improved.
In a possible implementation manner of the first aspect, the target position is a target control on a display interface of the electronic device; acquiring device coordinates of a target position of the electronic device in a device coordinate system, including: acquiring a control identifier corresponding to a target control; a control coordinate request for obtaining the equipment coordinates of the target control is sent to the electronic equipment, wherein the control coordinate request comprises the control identification of the target control; and receiving the equipment coordinates corresponding to the target control sent by the electronic equipment. Therefore, under the scene, the equipment coordinates corresponding to any control can be obtained directly through a tool installed on the electronic equipment, the equipment coordinates of the target position do not need to be written in the automation script, only the identification corresponding to the control needs to be written in the automation script, and the automation script does not need to be redeveloped for different types of electronic equipment with different physical sizes but unchanged control identification, so that the automation script developed by the scheme can be compatible with the electronic equipment with different physical sizes, namely, the compatibility of the automation script is improved.
In another possible implementation manner of the first aspect, the target position is a non-control position of a display interface of the electronic device; the acquiring the device coordinates of the target position of the electronic device in the device coordinate system includes: and receiving equipment coordinates corresponding to the target position, which are sent by the electronic equipment, wherein the equipment coordinates are obtained after the electronic equipment receives clicking operation on the target position. As can be seen, the scene requires device coordinates of the target location to be obtained in advance and written into an automation script.
In a further possible implementation manner of the first aspect, acquiring a first mapping relation between the mechanical coordinate system and the camera coordinate system includes: acquiring mechanical coordinates corresponding to at least two manipulator calibration points on the manipulator equipment; acquiring camera coordinates of each manipulator calibration point in a camera coordinate system through a camera; and obtaining a first mapping relation between the mechanical coordinate system and the camera coordinate system based on the mechanical coordinates and the camera coordinates corresponding to the at least two manipulator calibration points. Therefore, the calibration points are directly arranged on the manipulator equipment, so that the manipulator is not required to obtain the mechanical coordinates of the mechanical calibration points, decoupling between the manipulator equipment and the mechanical coordinates is realized, and meanwhile, the calibration efficiency is improved.
In another possible implementation manner of the first aspect, acquiring mechanical coordinates corresponding to at least two calibration points of the manipulator on the manipulator device includes: at least two calibration points are arranged on a clamp platform of the manipulator device, and mechanical coordinates of each calibration point are obtained based on device information of the manipulator device.
In a further possible implementation manner of the first aspect, acquiring the second mapping relationship between the camera coordinate system and the device coordinate system of the electronic device includes: acquiring device coordinates corresponding to at least two device calibration points on the electronic device; acquiring camera coordinates of each device calibration point in the camera coordinate system through a camera; and obtaining a second mapping relation between the camera coordinate system and the equipment coordinate system based on the equipment coordinates and the camera coordinates corresponding to the at least two equipment calibration points.
In a further possible implementation of the first aspect, the device calibration point is a calibration control on a display interface of the electronic device; acquiring device coordinates corresponding to at least two device calibration points on an electronic device, comprising: acquiring a control identifier corresponding to the calibration control; sending a control coordinate request to the electronic equipment, wherein the control coordinate request comprises the control identifier; and receiving the equipment coordinates corresponding to the calibration control sent by the electronic equipment.
In another possible implementation manner of the first aspect, the device calibration point is a calibration point in a preset picture displayed by the electronic device; the method for acquiring the equipment coordinates corresponding to the at least two equipment calibration points on the electronic equipment comprises the following steps: and obtaining the equipment coordinates of the calibration point in the equipment coordinate system based on the coordinates of the calibration point in the preset picture.
In a further possible implementation manner of the first aspect, acquiring, by the camera, camera coordinates of each device calibration point in the camera coordinate system includes: receiving an image of an electronic device captured by a camera, the image including a device calibration point; and identifying the device calibration points contained in the image based on the image identification technology, and determining the position coordinates of each device calibration point in the image to obtain the camera coordinates of the device calibration points.
In another possible implementation of the first aspect, the target location is a non-control location of a display interface of the electronic device; acquiring device coordinates of a target position of the electronic device in a device coordinate system, including: and receiving equipment coordinates corresponding to the target position sent by the electronic equipment, wherein the equipment coordinates are obtained after the electronic equipment receives clicking operation on the target position.
In a further possible implementation manner of the first aspect, the converting to obtain the mechanical coordinates of the target position based on the third mapping relation includes: and inputting the equipment coordinates of the target position into the mapping relation, and calculating to obtain the mechanical coordinates corresponding to the target position.
In a further possible implementation manner of the first aspect, the method further includes: controlling the manipulator equipment to move to the target position of the electronic equipment based on the mechanical coordinates of the target position; the control robot performs a corresponding operation at the target position.
In another possible implementation manner of the first aspect, controlling the movement of the manipulator device to the target position of the electronic device based on the mechanical coordinates of the target position includes: generating a motion control instruction based on the mechanical coordinates of the target position and the mechanical coordinates of the current position of the manipulator equipment; and sending the motion control instruction to the manipulator equipment so as to enable the manipulator equipment to respond to the motion control instruction and move from the current position to the target position.
In a second aspect, the present application also provides a computer device, the electronic device comprising: one or more processors, memory, and a touch screen; the memory is used for storing program codes; the processor is configured to execute program code to cause the electronic device to implement the manipulator system positioning method as described in any one of the possible implementations of the first aspect.
In a third aspect, the present application also provides a computer readable storage medium having instructions stored thereon that, when run on a computer device, cause the electronic device to perform the robotic system positioning method of any one of the first aspects.
It should be appreciated that the description of technical features, aspects, benefits or similar language in this application does not imply that all of the features and advantages may be realized with any single embodiment. Conversely, it should be understood that the description of features or advantages is intended to include, in at least one embodiment, the particular features, aspects, or advantages. Therefore, the description of technical features, technical solutions or advantageous effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and advantageous effects described in the present embodiment may also be combined in any appropriate manner. Those of skill in the art will appreciate that an embodiment may be implemented without one or more particular features, aspects, or benefits of a particular embodiment. In other embodiments, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of a manipulator clamp platform according to an embodiment of the present application;
fig. 2 is a schematic diagram of a corresponding image when a target electronic device is placed on a manipulator clamp platform provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a manipulator system according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a device coordinate system of a target electronic device according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a mechanical coordinate system, a camera coordinate system, and an equipment coordinate system according to an embodiment of the present application;
FIG. 6 is a flowchart of a method for positioning a manipulator system according to an embodiment of the present disclosure;
FIG. 7 is a schematic illustration of a calibration point on a fixture platform provided in an embodiment of the present application;
FIG. 8 is a schematic diagram of camera coordinates of a calibration point on a fixture platform provided in an embodiment of the present application;
FIG. 9 is a schematic diagram of device coordinates of a calibration point of a target electronic device provided in an embodiment of the present application;
FIG. 10 is a schematic diagram of camera coordinates of a calibration point of a target electronic device provided in an embodiment of the present application;
fig. 11 is a schematic diagram of camera coordinates corresponding to a calibration point on a target electronic device and a fixture platform according to an embodiment of the present application.
Detailed Description
The terms first, second, third and the like in the description and in the claims and drawings are used for distinguishing between different objects and not for limiting the specified sequence.
In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
For clarity and conciseness in the description of the following embodiments, a brief description of the related art will be given first:
the machine vision system uses a machine to replace human eyes to measure and judge, converts a shot object into an image signal through an image shooting device (such as a camera), further processes the image signal to obtain certain characteristics of the object, and further controls the on-site equipment action according to the characteristics.
Coordinate system calibration refers to calibration of a conversion relationship between two different coordinate systems, for example, in the application, the conversion relationship between a visual coordinate system (also referred to as a camera coordinate system), a mechanical coordinate system (i.e. a coordinate system of a manipulator clamp platform) and an equipment coordinate system is related, and the coordinate system calibration helps the robot (i.e. a manipulator system) to convert the obtained visual information, so as to complete subsequent control work, such as visual grabbing and the like.
The following describes a scenario of performing an automated test on a screen of an electronic device by using a manipulator system, where a positioning method of the manipulator system provided by the related art is as follows:
in the stage of developing an automation script, firstly, the electronic equipment is placed on a clamp platform of the manipulator, the manipulator is moved to a designated tested position, mechanical coordinates corresponding to the tested positions are recorded, and the mechanical coordinates are written into a test script.
And in the testing stage of the electronic equipment, executing an automation script, controlling the manipulator to move to the corresponding tested position according to the mechanical coordinates of the tested position in the testing script, and performing corresponding testing operation on the tested position.
It can be seen that in the stage of developing the automation script, the process relies on the manipulator device to obtain the mechanical coordinates corresponding to the detected position, so that the development efficiency of the automation script is low. In addition, if the physical size of the test equipment is changed, the mechanical coordinates of the tested position need to be obtained by the manipulator equipment again, and the coordinates of the test points in the automation script are modified, so that the compatibility of the automation script developed in the mode with electronic equipment with different physical sizes is poor.
Another manipulator system positioning method in the related art is as follows: and obtaining the mechanical coordinates of the detected position in the electronic equipment through the mapping relation between the camera coordinate system and the mechanical coordinate system, and further controlling the manipulator to move according to the mechanical coordinates corresponding to the detected position.
For example, to test whether the basic functions of an application installed in an electronic device meet requirements, corresponding controls (e.g., a setting control, a popup deletion control, a confirmation control, etc.) need to be controlled to perform corresponding operations. In this scenario, the icon of the control is relatively easy to identify through the image recognition technology, so that the mechanical coordinate of the corresponding position of the icon can be obtained by establishing a mapping relation between the mechanical coordinate system and the camera coordinate system, and the specific process is as follows:
And in the stage of developing the automatic test script, shooting a picture of the screen of the electronic equipment containing the icon to be tested by using a camera, further obtaining the picture of the icon to be identified from the picture, and storing the picture as a template picture.
When the automated test script is executed to perform automated testing on an electronic device, as shown in fig. 1, a target electronic device (such as a mobile phone, a tablet computer, a watch, etc.) is placed on a fixture platform of a manipulator. Then, an image of the target electronic device shown in fig. 2 is captured with a camera. After the icon object which is contained in the image and matched with the template picture is identified based on the image identification technology, camera coordinates corresponding to the icon object (such as a point P in FIG. 2) are obtained.
Further, according to the mapping relation between the mechanical coordinate system and the camera coordinate system, the camera coordinate of the P point is converted into the coordinate in the mechanical coordinate system, and the mechanical coordinate of the P point is obtained. Finally, the manipulator is controlled to execute corresponding operation based on the mechanical coordinates of the P point.
It can be seen that this approach requires taking a corresponding image with a camera when obtaining the mechanical coordinates of the device under test, resulting in a low test efficiency.
In order to solve the technical problems, the inventor of the application provides a positioning method of a manipulator system, wherein at least two calibration points are arranged on a clamp platform of a manipulator, namely, the mechanical coordinates of the calibration points are known. And shooting an image of the clamp platform by using a camera to obtain the coordinates of the calibration point on the clamp platform in a camera coordinate system, namely the camera coordinates of the calibration point. And obtaining the mapping relation between the mechanical coordinate system and the camera coordinate system through the mechanical coordinate and the camera coordinate corresponding to the calibration point on the clamp platform. And determining at least two calibration points on the measured electronic device, i.e. knowing the coordinates of the calibration points on the measured electronic device in the device coordinate system, i.e. the device coordinates. Then, the camera is utilized to shoot the image of the tested electronic equipment, and the camera coordinates of the standard point on the tested electronic equipment are obtained. And further obtaining the mapping relation between the equipment coordinate system and the camera coordinate system. Further, according to the mapping relation between the mechanical coordinate system and the camera coordinate system and the mapping relation between the camera coordinate system and the equipment coordinate system, the mapping relation between the equipment coordinate system and the mechanical coordinate system is obtained. Based on the mapping relation between the mechanical coordinate system and the equipment coordinate system, the equipment coordinate of any point on the tested electronic equipment can be directly converted into the mechanical coordinate. Finally, the manipulator is controlled to move to the corresponding position based on the mechanical coordinates of the point and the corresponding operation is performed.
Therefore, the manipulator positioning method provided by the scheme can directly control the manipulator based on the equipment coordinates of the detected position of the detected electronic equipment, and the automation script obtained by the scheme does not need to be written into the mechanical coordinates of the detected position, in other words, the development and debugging process of the automation script of the scheme is decoupled from the manipulator equipment, so that the development and debugging efficiency of the automation script is improved.
Meanwhile, the device coordinates of the detected position can be directly obtained by utilizing the scheme, the camera coordinates of the detected position do not need to be obtained by utilizing the camera, the time consumption of the process of obtaining the mechanical coordinates of the detected position is reduced, and the testing efficiency of the whole testing process is improved.
Further, in the scenario of performing automatic test on the tested piece of the tested device, the tested position is the position where the tested piece is located. Under the scene, the mechanical coordinates of the detected position of the electronic equipment are not required to be written in the automatic script of the scheme, but the control IDs corresponding to the detected parts are written in, and for the detected electronic equipment of the same type, such as mobile phones (or other electronic equipment) of two different types, the physical sizes are different, but the control IDs corresponding to the same detected part are not changed, so that the automatic script is not required to be modified for the detected electronic equipment, namely, the automatic script obtained by the scheme can be compatible with the detected equipment of different physical sizes, and the compatibility of the automatic script to the detected equipment of different physical sizes is improved.
For clarity and conciseness in the description of the embodiments below, the operation of the manipulator system will be described by using the manipulator system to perform automated testing on a mobile phone.
Referring to fig. 3, a schematic structural diagram of a manipulator system provided in an embodiment of the present application is shown, and as shown in fig. 3, the manipulator system includes: a robotic device, at least one camera (or video camera), a computer. Wherein, the manipulator
Including robotic devices include robots (or otherwise known as operating mechanisms) and clamp platforms (or otherwise known as support platforms).
It will be appreciated that the construction of the automatic assembling apparatus illustrated in this embodiment does not constitute a specific limitation on the apparatus. In other embodiments, the automated assembly device may include more or fewer components than shown, or certain components may be combined, certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
In the scenario of automated testing, the manipulator is used for controlling the electronic device to perform automated testing, and the clamp platform is used for fixing the electronic device. For example, the mobile phone to be tested is fixed on the clamp platform so as to perform automatic tests such as stability test, function test and power consumption test on the mobile phone.
In the present application, the target electronic device refers to an automatic test performed by a manipulator system, and the target electronic device may be a device such as a mobile phone, a tablet computer, a desktop, a laptop, a notebook, an Ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a handheld computer, a netbook, a personal digital assistant (Personal Digital Assistant, PDA), a wearable electronic device, a smart watch, etc., and the specific form of the electronic device is not particularly limited in the present application.
The camera is used for shooting images of the target electronic equipment and determining position coordinates of the target electronic equipment. The camera may be a CCD camera or a CMOS camera.
The computer is used for controlling the motion state of the manipulator, controlling the working state of the camera and carrying out image analysis on the image shot by the camera to obtain the coordinates of the target position in the image. The manipulator system positioning method provided by the application is operated in a computer shown in fig. 3.
And the computer sends a shooting instruction to the camera, and the camera executes the shooting instruction to perform corresponding shooting operation. The camera sends the shot image to the computer so that the computer can process the image to obtain the coordinates of the target position in the image and convert the coordinates of the target position into mechanical coordinates. Further, the computer generates a motion control instruction based on the mechanical coordinates of the target position and transmits the motion control instruction to the manipulator to control the manipulator to move to the target position and execute corresponding operations.
Taking the example that the target electronic device is a mobile phone, fig. 4 shows a coordinate system of the mobile phone, that is, a device coordinate system.
It is assumed that the above three coordinate systems (i.e., camera coordinate system, machine coordinate system, and device coordinate system) can be ensured to be in the same plane by improving the precision at the time of production of the manipulator, and as shown in fig. 5, the camera coordinate system, the machine coordinate system, and the device coordinate system in the same plane are shown.
As shown in fig. 5, the xoy coordinate system is a camera coordinate system, the x ' o ' y ' coordinate system is a machine coordinate system, and the x "o" y "coordinate system is a device coordinate system. The origin of the different coordinate systems and the positive direction of the coordinate axes are different.
As described above, the machine vision system is established based on the camera coordinate system, and the operation of the robot is based on the machine coordinate system, so that it is necessary to establish a mapping relationship between the camera coordinate system and the machine coordinate system. After the mapping relation between the two coordinate systems is obtained, if the coordinate of the position point in any coordinate system is known for the same position point, the coordinate of the position point in the other coordinate system can be converted according to the mapping relation between the two coordinate systems. For example, after obtaining the camera coordinates corresponding to the position points, the mechanical coordinates corresponding to the position points may be obtained by conversion according to the mapping relationship between the camera coordinates and the mechanical coordinates.
The following describes a procedure of a method for positioning a manipulator system according to an embodiment of the present application, which is executed in the computer in fig. 3, with reference to fig. 6. As shown in fig. 6, the positioning method of the manipulator system includes the following steps:
s110, acquiring mechanical coordinates of at least two calibration points A, B on a clamp platform of the manipulator.
In an example, two points may be fixed as calibration points on a fixture platform of the manipulator at the time of production of the manipulator, and mechanical coordinates of at least two calibration points may be marked at the time of shipment of the manipulator. For example, the mechanical coordinates of each of the calibration points may be marked directly on the fixture platform.
The number of the calibration points on the fixture platform can be two or more, which is not limited in the application.
In another example, two fixed points may be selected as calibration points on the fixture platform of the manipulator by a party using the manipulator, and the mechanical coordinates corresponding to the two calibration points may be determined according to the mechanical coordinate system of the manipulator.
For example, as shown in FIG. 7, the mechanical coordinates corresponding to two calibration points on the fixture platform are A (x' 1 ，y′ 1 )，B(x′ 2 ，y′ 2 )。
In an exemplary embodiment, image recognition accuracy may be improved by modifying the color of a calibration point on the fixture platform or increasing fluorescence.
S120, obtaining camera coordinates corresponding to each calibration point on the clamp platform.
An image of the fixture platform may be captured with a high definition camera, as shown in fig. 8, which is a schematic view of the image of the fixture platform captured by the camera, including the fixture platform, and the calibration points a and B on the fixture platform.
Further obtaining the coordinates of two standard points in the image in the camera coordinate system by image recognition technology, namely obtaining the camera coordinates corresponding to the standard points A and B in the image, such as A (x 1 ，y 1 )、B(x 2 ，y 2 )。
And S130, obtaining a mapping relation between the mechanical coordinate system and the camera coordinate system according to the mechanical coordinate and the camera coordinate corresponding to the calibration point on the clamp platform, or a first mapping relation.
If the camera coordinate and the mechanical coordinate corresponding to a certain point are known, a mapping relationship between the camera coordinate system and the mechanical coordinate system can be established, for example, the camera coordinate of a certain point is (x, y), and the coordinate of the point in the mechanical coordinate system is (x ', y'), and the mapping relationship between the camera coordinate system and the mechanical coordinate system can be expressed by the following formula (1):
a, b, c, d in equation (1) is an unknown parameter.
Camera coordinates (x 1 ，y 1 )、(x 2 ，y 2 ) And knowing the mechanical coordinates (x 'of A and B' 1 ，y′ 1 ) And (x' 2 ，y′ 2 ) Substituting the camera coordinates and the mechanical coordinates corresponding to the A, B two points into the formula (1) to obtain an equation set shown in the formula (2):
according to the equation set shown in the formula (2), a specific numerical value of the unknown parameter a, b, c, d can be obtained by solving, that is, if the camera coordinates of any point are known, the mechanical coordinates corresponding to the point can be obtained by converting the mapping relation. Similarly, if the mechanical coordinates corresponding to a point are known, the camera coordinates corresponding to the point can be obtained by conversion.
The mapping relation between the mechanical coordinate system and the camera coordinate system is only calculated again after the camera position is changed. On the premise that the camera position is not changed, the processes from S110 to S130 are only required to be executed once, and the first mapping relationship can be directly used subsequently.
The position of the camera may change, such as in a camera overhaul or replacement scenario. After the camera position is changed, the mapping relationship between the mechanical coordinate system and the camera coordinate system is recalculated according to the processes of S110 to S130.
And S140, acquiring coordinates (namely, device coordinates) of at least two calibration points on the target electronic device in a device coordinate system.
The embodiment of the application is described by taking the example that the target electronic device is a mobile phone. The number of the marked points on the target electronic device may be two or more, which is not limited in this application.
In the scenario of automatic testing on the screen of the mobile phone, two points on the mobile phone can be selected as the calibration points, for example, the positions of two controls on the display interface of the mobile phone can be selected as the calibration points. Alternatively, a picture containing a calibration point may be displayed on the mobile phone, the resolution of the picture is the same as that of the screen of the mobile phone, the picture may be a solid-color picture, and the color of the calibration point is different from that of the picture, for example, the picture may be white, and the color of the calibration point may be black.
For a scene with the calibration point being a control displayed on a mobile phone display interface, the UI automatic testing tool can be utilized to directly acquire the equipment coordinates corresponding to the controls. For example, device coordinates corresponding to the control may be obtained by installing a UI automation test tool (e.g., UI automation) in the electronic device.
For a scene where the calibration point is a calibration point in a picture, a picture containing the calibration point, i.e. the device coordinates of the calibration point in the known picture, may be generated according to the screen resolution of the device under test.
For example, as shown in fig. 9, the device coordinates corresponding to the two calibration points on the mobile phone are C (x″) 3 ，y″ 3 ) And D (x) 4 ，y″ 4 )。
S150, obtaining camera coordinates corresponding to the calibration points on the target electronic equipment.
Similar to the manner of acquiring the camera coordinates of the calibration point on the jig platform, the target electronic device is fixed at the fixed position of the jig platform of the manipulator, and then an image of the target electronic device is photographed by using the high-definition camera.
Further, the camera coordinates corresponding to the target point C, D in the image are obtained by image recognition technology, for example, as shown in fig. 10, the coordinates of the target point C, D in the camera coordinate system are C (x' 3 ，y′ 3 ) And D (x' 4 ，y′ 4 )。
And S160, obtaining a mapping relation between the camera coordinate system and the equipment coordinate system or a second mapping relation according to the equipment coordinate and the camera coordinate corresponding to the calibration point on the target electronic equipment.
For the same position point, if the coordinates of the point in the device coordinate system and the coordinates of the point in the camera coordinate system are known, a mapping relationship between the device coordinate system and the camera coordinate system can be established. For example, the mapping relationship may be expressed by the following formula (3):
a in the formula (3) 1 、b 1 、c 1 、d 1 Are all unknown parameters.
Substituting the device coordinates and the camera coordinates corresponding to the C, D two points on the target electronic device into the formula (3) to obtain an equation set shown in the following formula (4):
according to the equation set shown in the formula (4), the unknown parameter a can be obtained by solving 1 、b 1 、c 1 、d 1 If the camera coordinate corresponding to any point on the target electronic device is known, the device coordinate corresponding to the point can be obtained by converting the mapping relation. Similarly, if the device coordinates corresponding to any point on the target electronic device are known, the camera coordinates corresponding to the point can be obtained through conversion.
After the second mapping relationship between the camera coordinate system and the device coordinate system is obtained through the process described in S140-S160, the second mapping relationship may be directly used in the following process without changing the physical size of the electronic device, that is, S140-S160 need only be executed once.
In the case that the physical size of the electronic device is changed, the second mapping relationship may be updated according to the change of the electronic device, for example, the physical sizes of test devices of different models (for example, mobile phones of different models), or the physical sizes of test devices of different types (for example, mobile phones and tablet computers, smartwatches, etc.) may be different, and the corresponding device coordinate systems of the test devices are also different, so that the mapping relationship between the device coordinate system and the camera coordinate system needs to be established for the test devices of different physical sizes. That is, after the test equipment is replaced, the second mapping relation needs to be redetermined according to S140 to S160.
By the above steps, the parameters of the mapping relation between the mechanical coordinate system and the camera coordinate system, namely the specific numerical value of a, b, c, d, and the parameters of the mapping relation between the equipment coordinate system and the camera coordinate system, such as a, are obtained 1 、b 1 、c 1 、d 1 Specific values of (2). Further, according to the parameter values of the first mapping relation and the second mapping relation, a mechanical coordinate system and an equipment coordinate can be obtainedThe mapping relationship between the systems, or may be referred to as a third mapping relationship.
In another example, the fixture platform is marked with a calibration point A, B, and further the target electronic device with the calibration point C, D displayed thereon can be directly fixed in a fixed position on the fixture platform. Then, one image including the jig stage and the target electronic device is photographed with a camera, that is, an image as shown in fig. 11 is obtained.
As shown in fig. 11, the image includes the jig platform and the cellular phone placed on the jig platform, and includes the calibration point A, B on the jig platform, and the calibration point C, D on the cellular phone.
The camera coordinates corresponding to the calibration point A, B on the fixture platform and the camera coordinates corresponding to the calibration point C, D on the mobile phone can be obtained simultaneously by analyzing one image shown in fig. 11. The camera is not required to respectively shoot images of the clamp platform and the testing equipment, and the image analysis is respectively carried out to obtain the camera coordinates of the calibration point A, B and the calibration point C, D. The efficiency of the coordinate system calibration process is improved.
S170, obtaining a third mapping relation between the mechanical coordinate system and the device coordinate system based on the first mapping relation between the mechanical coordinate system and the camera coordinate system and the second mapping relation between the device coordinate system and the camera coordinate system.
Substituting the formula (1) into the formula (3) can obtain a third mapping relationship between the mechanical coordinate system and the equipment coordinate system, for example, the third mapping relationship is shown in the formula (5):
wherein a, b, c, d and a 1 、b 1 、c 1 、d 1 Are all known, therefore, parameter aa 1 -bb 1 ，a 1 b+ab 1 ，-ab 1 -a 1 b，a 1 c+b 1 d+c 1 And a 1 d-b 1 c+d 1 Specific values can be calculated.
It should be noted that, in the case where the position of the camera is not changed and the physical size of the electronic device is not changed, the process of obtaining the third mapping relationship in S170 is also only performed once, and the third mapping relationship may be directly used when the manipulator system is used subsequently. For example, when testing electronic devices with the same physical size, the process described in S110 to S170 may be executed only once to obtain the third mapping relationship, and then the third mapping relationship may be directly used when testing a plurality of electronic devices with the same physical size.
S180, obtaining equipment coordinates corresponding to the target position on the target electronic equipment, and converting the equipment coordinates of the target position into mechanical coordinates based on the third mapping relation.
Under the scene of utilizing the manipulator to operate the mobile phone screen and realizing automated test, the manipulator needs to be moved to the position of being tested and carries out corresponding test operation. For example, when performing touch performance test on a mobile phone screen, the mobile phone needs to move to a test point and perform touch operation on the screen; for another example, when performing a pressure test on a mobile phone screen, it is also necessary to move a manipulator to a test point and perform a pressing operation on the screen.
After obtaining the device coordinates corresponding to the detected position on the screen, the device coordinates of the detected position can be directly converted into mechanical coordinates according to the third mapping relation. Further, the manipulator is controlled to move to the corresponding detected position based on the mechanical coordinates corresponding to the detected position. In one scenario, where the detected position is a control on a mobile phone display interface, a UI automation test tool, such as a UI automation, may be installed in the target electronic device, and device coordinates of the tested control may be obtained by the UI automation test tool. In addition, after replacing other electronic equipment of the same type, for example, after testing some controls of a mobile phone of a certain model, continuously testing the controls of the mobile phone of another model, under the scene, the control IDs of the tested controls are unchanged, the equipment coordinates of the controls corresponding to the tested controls can be quickly obtained by using the UI automatic test tool, namely, the coordinates of the tested controls can be quickly obtained after replacing the electronic equipment, an automatic script is not required to be modified, and the compatibility of the automatic script is improved.
In another scenario, if the detected position is a position without a control, such as a blank position on the display interface of the mobile phone, in this scenario, the finger is positioned at the detected position of the mobile phone, and the device coordinate of the touched position can be obtained through the "pointer position" function of the electronic device, and the device coordinate of the detected position is written into the automation script. And at the stage of testing the tested equipment, converting the equipment coordinates of the tested position into mechanical coordinates based on the mapping relation between the equipment coordinate system and the mechanical coordinate system, controlling the manipulator to move to the tested position based on the mechanical coordinates, and executing corresponding test operation. It can be seen that this scenario also does not require the reliance on a robotic arm to develop an automation script.
And S190, generating a motion control instruction based on the mechanical coordinates of the target position, and sending the motion control instruction to the manipulator so that the manipulator moves to the target position.
In one example, the computer-generated motion control instructions may directly control the robot movement. For example, the computer generates a motion control command for controlling the movement of the manipulator based on the current position of the manipulator and the mechanical coordinates corresponding to the target position, and based on the mechanical coordinates of the two positions, for example, the motion control command is moved to the left by 300mm and then moved to the right by 500mm, and the generated motion control command is transmitted to the manipulator. And after the manipulator receives the motion control instruction and executes the instruction, the manipulator moves from the current position to the target position.
In another example, the computer-generated motion control instructions are used to inform the manipulator of the target position, e.g., after the computer obtains the mechanical coordinates of the target position, the motion control instructions are sent to the manipulator, the motion control instructions containing the target position. And the manipulator analyzes the motion control instruction to obtain the mechanical coordinates of the target position, and further, the manipulator generates a moving instruction according to the current position of the manipulator and the received target position and executes the moving instruction to move from the current position to the target position.
The control mode of the manipulator moving from the current position to the target position is not limited in the embodiment of the application.
S1100, sending a test instruction to the manipulator, so that the manipulator performs corresponding test operation on the target position on the target electronic equipment.
In an example, after the manipulator moves to the target position, the computer feeds back that the manipulator has moved to the target position, at this time, the computer sends a test instruction to the manipulator, for example, controls the manipulator to perform operations such as touching or pressing. And after the manipulator receives the test execution, analyzing and executing the test instruction, so as to realize the automatic test of the test equipment.
According to the manipulator system positioning method provided by the embodiment, at least two calibration points are arranged on the clamp platform of the manipulator, namely, the mechanical coordinates of the calibration points in the mechanical coordinate system are known. And shooting an image of the clamp platform by using a camera to obtain camera coordinates of a standard point on the clamp platform. And obtaining the mapping relation between the mechanical coordinate system and the camera coordinate system through the mechanical coordinates and the camera coordinates corresponding to the at least two calibration points on the clamp platform. Similarly, at least two calibration points may be determined on the target electronic device, i.e., device coordinates of the calibration points on the target electronic device are known. Then, an image of the target electronic device is captured with the camera, and camera coordinates of the calibration point on the target electronic device are obtained. And further obtaining the mapping relation between the equipment coordinate system and the camera coordinate system. Further, according to the mapping relation between the two coordinate systems, the mapping relation between the equipment coordinate system and the mechanical coordinate system is obtained. Finally, based on the mapping relation between the mechanical coordinate system and the equipment coordinate system, the equipment coordinate of any point on the target electronic equipment can be directly converted into the mechanical coordinate. Finally, the manipulator is controlled to move to the corresponding position based on the mechanical coordinates of the point and the corresponding operation is performed. Therefore, the mechanical arm can be directly controlled based on the input equipment coordinates of the target electronic equipment by utilizing the scheme, and the automatic script obtained by the scheme does not need to be written into the mechanical coordinates of the measured position, in other words, the development and debugging process of the automatic script of the scheme is decoupled from the mechanical arm equipment, so that the development and debugging efficiency of the automatic script is improved.
Meanwhile, the device coordinates of the detected position can be directly obtained by utilizing the scheme, the camera coordinates of the detected position do not need to be obtained by utilizing the camera, the time consumption of the process of obtaining the mechanical coordinates of the detected position is reduced, and the testing efficiency of the whole testing process is improved.
Further, after the camera position is changed, the camera coordinate system is changed, and the first mapping relation between the mechanical coordinate system and the camera coordinate system, the second mapping relation between the camera coordinate system and the equipment coordinate system, and the third mapping relation between the mechanical coordinate system and the equipment coordinate system are changed. Similarly, after the test equipment is replaced, the physical size of the test equipment is changed, the equipment coordinate system is changed, the second mapping relation between the equipment coordinate system and the camera coordinate system is changed, and the third mapping relation between the equipment coordinate system and the mechanical coordinate system is changed. By means of the scheme, at least two calibration points are respectively arranged on the clamp platform and the testing equipment of the manipulator in advance, and the first mapping relation between the mechanical coordinate system and the camera coordinate system can be rapidly and automatically marked based on the calibration points on the clamp platform. The second mapping relationship between the device coordinate system and the camera coordinate system can be quickly and automatically marked based on the calibration points on the test device. After updating the first mapping relation and the second mapping relation, the third mapping relation can be updated rapidly. Therefore, the scheme can quickly realize automatic calibration among three coordinate systems, and improves the efficiency of the coordinate system calibration process.
Further, in the scenario of performing automatic test on the tested piece of the tested device, the tested position is the position where the tested piece is located. Under the scene, the mechanical coordinates of the detected position of the electronic equipment are not required to be written in the automatic script of the scheme, but the control IDs corresponding to the detected parts are written in, and for the detected electronic equipment of the same type, such as mobile phones (or other electronic equipment) of two different types, the physical sizes are different, but the control IDs corresponding to the same detected part are not changed, so that the automatic script is not required to be modified for the detected electronic equipment, namely, the automatic script obtained by the scheme can be compatible with the detected equipment of different physical sizes, and the compatibility of the automatic script to the detected equipment of different physical sizes is improved.
From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above. The specific working processes of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which are not described herein.
In the several embodiments provided in this embodiment, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present embodiment may be integrated in one processing unit, each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present embodiment may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in the respective embodiments. And the aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic or optical disk, and the like.
The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (13)
1. A method for positioning a manipulator system, comprising:
acquiring a first mapping relation between a mechanical coordinate system and a camera coordinate system;
obtaining a second mapping relation between a camera coordinate system and a device coordinate system of the electronic device based on device coordinates and camera coordinates corresponding to at least two device calibration points of the electronic device, wherein the device calibration points are calibration controls on a display interface of the electronic device or calibration points in a preset picture displayed by the electronic device;
obtaining a third mapping relation between the equipment coordinate system and the mechanical coordinate system based on the first mapping relation and the second mapping relation;
and acquiring equipment coordinates of the target position of the electronic equipment in the equipment coordinate system, and converting the equipment coordinates into mechanical coordinates of the target position based on the third mapping relation.
2. The method of claim 1, wherein the obtaining a first mapping between a mechanical coordinate system and a camera coordinate system comprises:
acquiring mechanical coordinates corresponding to at least two manipulator calibration points on the manipulator equipment;
acquiring camera coordinates of each manipulator calibration point in the camera coordinate system through a camera;
And obtaining a first mapping relation between the mechanical coordinate system and the camera coordinate system based on the mechanical coordinates and the camera coordinates corresponding to the at least two manipulator calibration points.
3. The method of claim 2, wherein the acquiring mechanical coordinates corresponding to at least two robot calibration points on the robot device comprises:
and at least two calibration points are arranged on the clamp platform of the manipulator equipment, and the mechanical coordinates of each calibration point are obtained based on the equipment information of the manipulator equipment.
4. The method according to claim 1 or 2, wherein the obtaining a second mapping relationship between a camera coordinate system and a device coordinate system of the electronic device based on the device coordinates and the camera coordinates corresponding to the at least two device calibration points of the electronic device comprises:
acquiring equipment coordinates corresponding to at least two equipment calibration points on the electronic equipment;
acquiring camera coordinates of each device calibration point in the camera coordinate system through a camera;
and obtaining a second mapping relation between the camera coordinate system and the equipment coordinate system based on the equipment coordinates and the camera coordinates corresponding to the at least two equipment calibration points.
5. The method of claim 4, wherein the device calibration point is a calibration control on a display interface of the electronic device; acquiring device coordinates corresponding to at least two device calibration points on the electronic device, including:
acquiring a control identifier corresponding to the calibration control;
sending a control coordinate request to the electronic equipment, wherein the control coordinate request comprises the control identifier;
and receiving the equipment coordinates corresponding to the calibration control sent by the electronic equipment.
6. The method of claim 4, wherein the device calibration point is a calibration point in a preset picture displayed by the electronic device; acquiring device coordinates corresponding to at least two device calibration points on the electronic device, including:
and obtaining the equipment coordinates of the calibration point in the equipment coordinate system based on the coordinates of the calibration point in the preset picture.
7. The method of claim 4, wherein the obtaining, by the camera, camera coordinates of each device calibration point in the camera coordinate system comprises:
receiving an image of the electronic device captured by a camera, the image comprising the device calibration point;
And identifying the equipment calibration points contained in the image based on an image identification technology, and determining the position coordinates of each equipment calibration point in the image to obtain the camera coordinates of the equipment calibration points.
8. The method of claim 1, wherein the target location is a non-control location of a display interface of the electronic device; the acquiring the device coordinates of the target position of the electronic device in the device coordinate system includes:
and receiving equipment coordinates corresponding to the target position, which are sent by the electronic equipment, wherein the equipment coordinates are obtained after the electronic equipment receives clicking operation on the target position.
9. The method of claim 1, wherein converting to obtain mechanical coordinates of the target location based on the third mapping relationship comprises:
and inputting the equipment coordinates of the target position into the mapping relation, and calculating to obtain the mechanical coordinates corresponding to the target position.
10. The method according to claim 1, wherein the method further comprises:
controlling the manipulator equipment to move to the target position of the electronic equipment based on the mechanical coordinates of the target position;
And controlling the manipulator to execute corresponding operation at the target position.
11. The method of claim 10, wherein controlling the movement of the robotic device to the target location of the electronic device based on the mechanical coordinates of the target location comprises:
generating a motion control instruction based on the mechanical coordinates of the target position and the mechanical coordinates of the current position of the manipulator equipment;
and sending the motion control instruction to the manipulator equipment so as to enable the manipulator equipment to respond to the motion control instruction and move from the current position to the target position.
12. A computer device, the electronic device comprising: one or more processors, memory, and a touch screen; the memory is used for storing program codes; the processor is configured to execute the program code to cause the electronic device to implement the manipulator system positioning method of any one of claims 1 to 11.
13. A computer readable storage medium having instructions stored thereon which, when run on a computer device, cause the electronic device to perform the robotic system positioning method of any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210286905.4A CN114670194B (en) | 2022-03-22 | 2022-03-22 | Positioning method and device for manipulator system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210286905.4A CN114670194B (en) | 2022-03-22 | 2022-03-22 | Positioning method and device for manipulator system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114670194A CN114670194A (en) | 2022-06-28 |
| CN114670194B true CN114670194B (en) | 2023-06-27 |
Family
ID=82074025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210286905.4A Active CN114670194B (en) | 2022-03-22 | 2022-03-22 | Positioning method and device for manipulator system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114670194B (en) |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064557A (en) * | 2013-01-05 | 2013-04-24 | 联信摩贝软件（北京）有限公司 | Method and system for inputting information to electronic equipment on basis of coordinates and acceleration of touch screen |
| CN105631875A (en) * | 2015-12-25 | 2016-06-01 | 广州视源电子科技股份有限公司 | Method and system for determining mapping relations between camera coordinates and arm gripper coordinates |
| CN105867433A (en) * | 2016-03-31 | 2016-08-17 | 纳恩博（北京）科技有限公司 | Moving control method, moving electronic device and moving control system |
| CN109187637B (en) * | 2018-07-31 | 2020-05-19 | 中国科学院自动化研究所 | Workpiece defect measuring method and system based on thermal infrared imager |
| CN110488751B (en) * | 2018-08-29 | 2022-08-19 | 中山大学 | Graphite tray visual positioning system of automatic process line |
| CN109974584B (en) * | 2019-04-12 | 2020-03-20 | 山东大学 | Calibration system and calibration method for auxiliary laser osteotomy robot |
| CN110238845B (en) * | 2019-05-22 | 2021-12-10 | 湖南视比特机器人有限公司 | Automatic hand-eye calibration method and device for optimal calibration point selection and error self-measurement |
| CN110640745B (en) * | 2019-11-01 | 2021-06-22 | 苏州大学 | Vision-based robot automatic calibration method, equipment and storage medium |
| CN111775154B (en) * | 2020-07-20 | 2021-09-03 | 广东拓斯达科技股份有限公司 | Robot vision system |
| CN112669383B (en) * | 2020-12-30 | 2023-04-07 | 上海新时达机器人有限公司 | Tracking method and camera calibration method of assembly line tracking system |
| CN113592959B (en) * | 2021-08-17 | 2023-11-28 | 北京博视智动技术有限公司 | Visual processing-based membrane lamination method and system |
| CN113643384B (en) * | 2021-10-12 | 2022-02-08 | 深圳荣耀智能机器有限公司 | Coordinate system calibration method, automatic assembly method and device |
| CN113977637A (en) * | 2021-12-03 | 2022-01-28 | 深圳市超准视觉科技有限公司 | Robot vision identification grabbing system and method applicable to non-precision work bin |
-
2022
- 2022-03-22 CN CN202210286905.4A patent/CN114670194B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114670194A (en) | 2022-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10780585B2 (en) | Robot and electronic device for performing hand-eye calibration | |
| EP2963513B1 (en) | Teaching apparatus and robot system | |
| US11498220B2 (en) | Control system and control method | |
| EP3866112A2 (en) | Method, apparatus, device, storage medium and program for three-dimensional reconstruction | |
| US7377650B2 (en) | Projection of synthetic information | |
| CN110706147B (en) | Image processing environment determination method, device, electronic equipment and storage medium | |
| CN114080590A (en) | Robotic bin picking system and method using advanced scanning techniques | |
| WO2022205625A1 (en) | Dispensing path generation method and apparatus, electronic device, and storage medium | |
| CN113840695B (en) | Calibration inspection assembly, robot system, inspection method and calibration method | |
| CN110582733B (en) | Method and device for estimating systematic errors of a commissioning tool of an industrial robot | |
| CN114670194B (en) | Positioning method and device for manipulator system | |
| CN100557558C (en) | Optical point device and data transmission method thereof | |
| US20180224825A1 (en) | Image processing system, image processing device, method of reconfiguring circuit in fpga, and program for reconfiguring circuit in fpga | |
| CN110322484B (en) | Calibration method and system for multi-device shared augmented reality virtual space | |
| CN209765441U (en) | Multi-mode dynamic gesture recognition device | |
| CN111898489A (en) | Method and device for marking palm pose, electronic equipment and storage medium | |
| CN113495162A (en) | Control system of automatic optical detection equipment | |
| CN114343483B (en) | Control method, device, equipment and storage medium for movable object | |
| CN116117810B (en) | Method and device for secondary calibration after data conversion among different robots | |
| US20230249345A1 (en) | System and method for sequencing assembly tasks | |
| CN111208417B (en) | Test method, device, equipment and medium | |
| CN118052890A (en) | Real-time hand-eye calibration method for multitasking scene | |
| CN117850639A (en) | Configuration management method and system for visual guidance positioning system | |
| JP6264742B2 (en) | Apparatus, method, program | |
| CN115541275A (en) | Simulation test system, method, device, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |