WO2023217047A1

WO2023217047A1 - Positioning method and apparatus, and electronic device and readable storage medium

Info

Publication number: WO2023217047A1
Application number: PCT/CN2023/092579
Authority: WO
Inventors: 李明洋; 许雄; 云鹏辉; 杨帆; 戚祯祥; 吴为; 汪辉
Original assignee: 节卡机器人股份有限公司
Priority date: 2022-05-07
Filing date: 2023-05-06
Publication date: 2023-11-16
Also published as: CN114820794A

Abstract

Provided in the present application are a positioning method and apparatus, and an electronic device and a readable storage medium. The method comprises: calculating first coordinates of a target point location of a target device by means of a visual algorithm; calculating second coordinates of the target point location of the target device by means of a point laser algorithm; and obtaining target coordinates of the target point location of the target device according to the first coordinates and the second coordinates. In the embodiments of the present application, target coordinates of a target point location of a target device are calculated by means of two coordinate calculation modes, that is, a visual algorithm and a laser point algorithm, and the actual coordinates of the target point location of the target device can be calculated simply by means of images and some calculation rules, such that the calculation of the target point location coordinates of the target point location of the target device is more flexible.

Description

Positioning methods, devices, electronic equipment and readable storage media

Cross-references to related applications

This application requests the priority of the Chinese patent application with application number 202210491867.6 and titled "Positioning method, device, electronic equipment and readable storage medium" submitted to the State Intellectual Property Office of China on May 7, 2022, and its entire contents incorporated herein by reference.

Technical field

The present application relates to the field of positioning technology, specifically, to a positioning method, device, electronic equipment and readable storage medium.

Background technique

In the field of robot automation, there is usually a certain deviation between the actual coordinates of the item to be picked up and the coordinates of the item to be picked up in the coordinate system of the robot. Therefore, it is necessary to calculate the actual position of the object to be taken before taking it. For example, in terms of nucleic acid testing, current nucleic acid testing generally relies on manual testing by medical personnel. The detection efficiency is low and the detection process carries a high risk of infection. Therefore, it is very important to realize automated nucleic acid detection. However, in actual work, there will always be a certain deviation between the actual placement of the item to be picked up and the robot's built-in coordinate system. Before the robot picks up the detection item, it is necessary to map the coordinates of the item to be picked up to the coordinate system where the robot is located. , to get the actual coordinates of the item to be picked up. However, at present, the actual coordinates of the items to be picked up are generally obtained through mechanical hard positioning, which is inflexible.

Contents of the invention

In view of this, embodiments of the present application provide a positioning method, device, electronic device, and readable storage medium. Ability to flexibly position items for pickup.

Embodiments of the present application provide a positioning method that calculates the first coordinates of a target point of a target device through a visual algorithm, where the first coordinates are the coordinates of the target point on a first plane in a calibration coordinate system; The second coordinates of the target point of the target device are calculated through a point laser algorithm. The second coordinates are the coordinates of the target point on the first plane in the calibration coordinate system; according to the first coordinates The target coordinates of the target point of the target device are determined with the second coordinates.

In the above implementation process, the coordinates of the target point on the first plane in the calibration coordinate system are calculated through the visual algorithm, and then the coordinates of the target point on the second plane in the calibration coordinate system are calculated through the point laser algorithm. Each coordinate value of the target point is accurately calculated to obtain the precise target coordinates of the target point. In addition, when performing visual algorithms, the real-time image data of the target device is obtained through the image acquisition device for processing. When processing through point laser, partial coordinates of the reference point are directly obtained through the laser device for processing. Since both laser equipment and image acquisition equipment can be flexibly installed and applied, and are suitable for various scenarios, the visual algorithm and the point laser algorithm are both relatively flexible. The positioning method increases the flexibility of positioning methods through the visual algorithm and point laser algorithm.

In conjunction with the above embodiment, the embodiment of the present application provides a first possible implementation manner of the above embodiment, wherein: calculating the first coordinate of the target point of the target device through a visual algorithm may include: collecting the collected The image of the target device is matched with an image template, and the image template is the image of the target device calibrated by the coordinate system; if the match is successful, the marker points are processed through a feature matching algorithm to output the actual coordinates of the marker points, and the marker points is the set mark point position on the target device, and the actual coordinates of the mark point are the coordinates of the mark point on the image of the target device; the coordinates of the target device are calculated according to the actual coordinates of the mark point. The first coordinate of the target point.

In the above implementation process, by comparing the image of the target device with the image template, the matching situation between the image of the target device and the image template is determined. When the image of the target device and the image template are successfully matched, the image template can be determined to be the target device. image template. Then the actual coordinates of the marker point are calculated through the feature matching algorithm to obtain the actual coordinates of the marker point. Since there is a fixed positional relationship between the marker point and the target point, after the actual coordinate value of the marker point is determined, based on the actual coordinates of the marker point and The actual coordinates of the target point can be calculated based on the positional relationship of the target point. The coordinate value of the target point is calculated by using the coordinate value of the fixed marker point. By matching the image information to the coordinate information, the calculation method is simplified, and the calculation efficiency and accuracy are improved.

In conjunction with the first possible implementation of the above embodiment, the embodiment of the present application provides a second possible implementation of the above embodiment, wherein: the first possible implementation of the target device is calculated based on the actual coordinates of the marker point. The coordinates may include: a coordinate difference calculated based on the actual coordinates of the marker point and the coordinates of the marker point template, where the marker point template coordinates are the coordinates of the marker point on the image template; determined by the coordinate difference The first coordinate of the target point of the target device.

In the above implementation process, the coordinate difference is obtained by calculating the actual coordinates of the marker point and the coordinates of the marker point template. Since the positions of the marker point and the target point are relatively fixed, the coordinate difference of the marker point can also be directly used as the target. The coordinate difference of the point directly calculates the actual coordinate of the target point. The calculation of the actual coordinates of the target point is based on the coordinates of the marked point and the coordinate difference. They are all calculated and processed based on the coordinates and are not restricted by usage scenarios, equipment, etc., thus increasing the flexibility of this positioning method. sex.

In conjunction with a possible implementation of the above embodiment, the embodiment of the present application provides another possible implementation of the above embodiment, wherein the coordinate difference of the target point of the target device is determined. The first coordinate may include: compensating the coordinate difference to the template coordinate of the target point of the target device to obtain the first coordinate of the target point of the target device, where the The template coordinates are the coordinates of the target point in the image template.

In conjunction with the second possible implementation manner of the above embodiment, the embodiment of the present application provides a third possible implementation manner of the above embodiment, wherein the target point position of the target device is calculated through a point laser algorithm. Before the second coordinates, the method may further include: obtaining multiple reference points of the target device; calculating multiple reference points through a visual algorithm. a first coordinate axis value of the reference point and a second coordinate axis value of the reference point.

In the above implementation process, on the basis of the visual algorithm calculating the first coordinate of the target point, the first coordinate axis value and the second coordinate axis value of the reference point are then calculated. Since the previous calculation of the target point has already been A series of data such as coordinate difference. When calculating the first coordinate axis value and the second coordinate axis value of the reference point, you only need to compensate the coordinate difference into the template coordinate of the reference point, which simplifies the reference point. The calculation of the first coordinate axis value and the second coordinate axis value improves the calculation accuracy and efficiency.

In conjunction with the second possible implementation of the above embodiment, the embodiment of the present application provides a fourth possible implementation of the above embodiment, wherein the target point of the target device is obtained through a point laser algorithm The second coordinate may include: obtaining the third coordinate axis value of a plurality of the reference points and the third coordinate axis value of the target point through a point laser; The coordinates and the third coordinate axis value of the target point are used to obtain the second coordinates of the target point of the target device, and the reference point coordinates of the reference point include the first coordinate axis of the reference point. value, the second coordinate axis value of the reference point and the third coordinate axis value of the reference point.

In the above implementation process, since the visual algorithm can only calculate the value on one plane, it is difficult to calculate the value in the height direction. Using a point laser to obtain the height direction values of multiple reference points and target points can make up for this. The deficiencies of the vision algorithm are used to obtain the coordinate values of the reference point and target point in all directions. The second rotation angle and the third rotation angle of the target point can be further calculated based on the complete coordinate value of the reference point. Through the combination of the above two methods, the complete coordinate value of the target point is calculated, which improves the accuracy of the actual coordinates of the target point. Moreover, the image acquisition equipment and laser equipment can be easily installed and replaced when the scene is frequently changed. The cost is low, so while improving the flexibility of using this positioning method, it also reduces the cost.

In conjunction with the fourth possible implementation manner of the above embodiment, the embodiment of the present application provides a fifth possible implementation manner of the above embodiment, wherein the obtaining multiple reference points of the target device may include: obtaining a third A reference point, a second reference point and a third reference point. The first reference point and the second reference point are at the same position on the first coordinate axis. The first reference point and the third reference point are on The position of the second coordinate axis is the same.

In the above implementation process, by setting the first coordinate axes of the first reference coordinate and the second reference coordinate to be the same, the first reference coordinate and the second coordinate axis of the third reference coordinate are the same. Since the selected first reference coordinates, second reference coordinates and third reference coordinates are all related to each other, they are representative, and they are all points that are easy to calculate. Therefore, calculations can be performed more simply and conveniently, calculations are simplified, and calculation efficiency and accuracy are improved.

In conjunction with the fifth possible implementation manner of the above-mentioned embodiment, the embodiment of the present application provides a sixth possible implementation manner of the above-mentioned embodiment, wherein the method is based on the plurality of reference point coordinates and the target point position. Obtaining the second coordinate of the target point of the target device with the third coordinate axis value may include: calculating the first included angle according to the first reference point coordinate and the second reference point coordinate, the The first included angle is the included angle between the line connecting the first reference point and the second reference point and the second coordinate axis; according to the coordinates of the first reference point and the coordinates of the third reference point count Calculate the second included angle, which is the included angle between the line connecting the first reference point and the third reference point and the first coordinate axis; according to the first included angle , the second included angle and the third coordinate axis value of the target point are used to obtain the second coordinates of the target point of the target device.

In the above implementation process, since the coordinates of the first reference point and the coordinates of the second reference point are at the same position on the first coordinate axis, it can be known according to the mathematical relationship between the line connecting the first reference point and the second reference point and the second coordinate axis. The angle between is the second rotation angle of the target point. Since the coordinates of the first reference point and the coordinates of the third reference point are at the same position on the second coordinate axis, the relationship between the first reference point and the third reference point can be known according to the mathematical relationship. The angle between the connection line and the first coordinate axis is the third rotation angle of the target point. By calculating the angle between the connection line and the coordinate axis of the reference point, the second rotation angle and the third rotation angle of the target point can be further obtained. Three angles of rotation. Since the entire process only involves the calculation and conversion of numerical values, it can be applied in various scenarios, improving the flexibility of this method.

An embodiment of the present application further provides a positioning device, which may include: a first calculation module configured to calculate the first coordinates of a target point of the target device through a visual algorithm; Two calculation modules, the second calculation module is configured to calculate the second coordinates of the target point of the target device through a point laser algorithm; a processing module, the processing module is configured to calculate the second coordinate of the target point of the target device according to the The first coordinate and the second coordinate obtain the target coordinate of the target point of the target device. An embodiment of the present application also provides an electronic device. The electronic device may include: a processor and a memory. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the When the machine-readable instructions are executed by the processor, the steps of the method in some of the above-mentioned embodiments or any possible implementation of some of the above-mentioned embodiments are performed.

Embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the computer program executes some of the above embodiments, or any one of the above embodiments. The steps of the positioning method in a possible implementation manner.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, embodiments are given below and described in detail with reference to the attached drawings.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore do not It should be regarded as a limitation of the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a block diagram of an execution device provided by an embodiment of the present application;

Figure 2 is a flow chart of the positioning method provided by the embodiment of the present application;

Figure 3 is a schematic diagram of the coordinate system provided by the embodiment of the present application;

Figure 4 is a flow chart of step 201 of the positioning method provided by the embodiment of the present application;

Figure 5 is a flow chart of step 202 of the positioning method provided by the embodiment of the present application;

Figure 6 is a schematic diagram of the functional modules of the positioning device provided by the embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings in the embodiments of this application.

It should be noted that similar reference numerals and letters represent similar items in the following figures, therefore, once an item is defined in one figure, it does not need further definition and explanation in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", etc. are only used to differentiate the description and cannot be understood as indicating or implying relative importance.

With the rapid development of automation, more and more fields require machinery to replace simple manual labor, such as in the fields of cargo handling, material taking, material movement, and express sorting. By using robots to perform tasks such as sorting, placing, moving and handling goods, materials or express delivery, it can effectively replace manual labor. However, if a robot is used for picking and placing operations, it is necessary to ensure that the actual position of the item is consistent with the robot's internal coordinate system, so that the robot can accurately obtain the target item during picking and other operations.

At present, the common positioning method is mechanical hard positioning. Mechanical hard positioning is mainly based on various positioning design objects such as positioning plates, positioning pins, and material setting pins, which are used for individual materials. Since the above-mentioned positioning parts for mechanical hard positioning are designed based on the properties of the material to be positioned, after the material is replaced, the original positioning parts are not suitable for the new material, so new positioning parts need to be redesigned. The above-mentioned mechanical hard positioning method has a strong restrictive effect on materials, and when there are more materials, the number of positioning parts that need to be replaced is also larger, which is not only easy to cause waste, but also inflexible in use. It needs to be based on the material. Change the positioning parts in a timely manner.

In view of this, the inventor of the present application proposes a new positioning method, which uses a visual algorithm to obtain an image of the item to be located, matches the image and calculates a part of the coordinates of the item to be located, and then obtains the reference through a point laser algorithm The coordinates of the point are calculated based on the coordinates of the reference point to obtain the coordinates of another part of the item to be located. Through the combination of the image acquisition equipment and the laser equipment, image data and partial point data are obtained, and then the actual coordinates of the target device are calculated based on the obtained image data and partial point data. Since both the image acquisition equipment and the laser equipment can be flexible The positioning method of this application can be flexibly used in various scenarios.

In order to facilitate understanding of this embodiment, the operating environment for executing a positioning method disclosed in the embodiment of this application is first introduced in detail.

The positioning system of this application may include an execution device, a target device and a processing device. The target device is used to place objects, which can be a loading platform, a loading trolley, a robot, an automatic navigation device, etc. The execution device is used to perform automatic operations, and can be a robot, a robotic arm, a stacker truck, a filling machine, etc. The processing device is used to determine the actual location of the target point, and can be a computer, programmable controller, tablet computer, smart phone, personal digital assistant (personal digital assistant, PDA), etc. The processing device may be communicatively connected to the execution device via a network for data communication or Interaction. The processing device may also communicate with the target device through a network for data communication or interaction. The target device can communicate with the execution device through the network for data communication or interaction.

It can be understood that the execution device and the processing device can be two devices that are independent of each other, or they can be two parts of the same physical device that correspond to different functions. For example, the processing device can be a device that performs automatic tasks and is responsible for External communication, interaction, and systems that support programming development, and the execution device can be the part of the device that performs automatic tasks that is responsible for automatic tasks. The target device and the processing device can be two independent devices, or they can be two parts of the same physical device corresponding to different functions. For example, the processing device can be a device that performs automatic tasks and is responsible for communicating and interacting with the outside. As well as systems that support programming development, the target device may be a part for placing items. The target device, the execution device and the processing device can be three independent devices, or they can be three parts of the same physical device corresponding to different functions. For example, the processing device can be a device that performs an automatic job and is responsible for communicating with external devices. For systems that communicate, interact, and support programming development, the target device can be the part used to place objects, and the execution device can be the part responsible for automatic operations in the device that performs automatic operations.

If the target device, the execution device and the processing device are three independent devices. Then a loading platform is provided on the target device for placing multiple items. The processing device is provided with relevant programs to execute the positioning method in this application, and sends the obtained actual coordinates of the target point on the target device to the execution device. After receiving the actual coordinates of the target point, the execution device executes the fetching task on the target device based on the actual coordinates of the target point.

If the execution device and the processing device are two parts of the same physical device corresponding to different functions. Then a loading platform is provided on the target device for placing multiple items. The execution device is equipped with relevant programs to execute the positioning method in this application, obtain the actual coordinates of the target point on the target device, and perform the fetching task on the target device according to the actual coordinates of the target point.

If the target device and the processing device are two parts of the same physical device corresponding to different functions. Then a stage is provided on the target device for placing multiple items, and the target device is provided with relevant programs to execute the positioning method in this application, and the actual coordinates of the target point on the target device are obtained and sent to the execution equipment. After receiving the actual coordinates of the target point, the execution device executes the fetching task on the target device based on the actual coordinates of the target point.

The positioning system is also provided with an image acquisition device and a laser device. The image acquisition device is configured to acquire image information such as an image template and an image of the target device. It can be set on the execution device or the target device. This laser device is used to obtain partial coordinate values of the point, and can be set on the execution device or the target device. The image acquisition device can communicate with the processing device through a network for data communication or interaction. The laser device can also be communicated with the processing device through a network for data communication or interaction.

It can be understood that the positioning method of the present application can be applied to nucleic acid detection robots, loading robots, handling robots, hotel robots, service robots, etc.

For example, if the positioning method is suitable for a nucleic acid detection robot, the target device can be a carrier. The object table is equipped with an area dedicated to placing test samples, tip boxes, liquid holders, etc. When performing nucleic acid testing, before the nucleic acid testing robot takes the test sample, the processing device collects the image of the target device through the image acquisition device, and matches it with the image template of the target device stored internally. If the match is successful, the collected image is deemed It is indeed an image of the target device. The feature matching algorithm is further used to process the marker points to obtain the actual coordinates of the marker points. Since the marker points maintain a fixed positional relationship with other points on the target device, after obtaining the actual coordinates of the marker points, the actual coordinates of the marker points can be obtained. The coordinates are calculated to obtain a part of the coordinates of the target sample. On the other hand, the laser equipment obtains the coordinates of the reference point and the coordinates of another part of the target sample through a point laser, and calculates the coordinates of another part of the target sample based on the mathematical relationship and the coordinates of the reference point. The actual coordinates of the target sample are determined based on the two parts of the coordinates, and the actual coordinates are sent to the nucleic acid detection robot. The nucleic acid detection robot controls the gripper to move to the actual coordinates of the target sample according to the actual coordinates to perform the picking operation.

In order to facilitate the understanding of this embodiment, the processing device for executing a positioning method disclosed in the embodiment of this application will be introduced in detail below.

As shown in Figure 1, it is a block diagram of a processing device 100. The processing device 100 may include a memory 111 and a processor 112 . Persons of ordinary skill in the art can understand that the structure shown in FIG. 1 is only illustrative and does not limit the structure of the processing device 100 . For example, processing device 100 may also include more or fewer components than shown in FIG. 1 , or have a different configuration than shown in FIG. 1 .

The above-mentioned components of the memory 111 and the processor 112 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these components may be electrically connected to each other through one or more communication buses or signal lines. The above-mentioned processor 112 is configured to execute executable modules stored in the memory.

Among them, the memory 111 can be, but is not limited to, random access memory (Random Access Memory, referred to as RAM), read-only memory (Read Only Memory, referred to as ROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM) ), Erasable Programmable Read-Only Memory (EPROM for short), Electrically Erasable Programmable Read-Only Memory (EEPROM for short), etc. The memory 111 is configured to store a program, and the processor 112 executes the program after receiving the execution instruction. The method executed by the process-defined processing device 100 disclosed in any embodiment of the present application can be Applied in the processor 112, or may be implemented by the processor 112.

The above-mentioned memory 111 may be configured to store data such as image templates, marker point template coordinates, and images of the target device, actual coordinates of the marker points, coordinate differences, and reference point coordinates.

The above-mentioned processor 112 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 112 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP). ), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logical block diagram disclosed in the embodiment of this application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The above-mentioned processor 112 is configured to obtain information such as image templates, marker point template coordinates, and reference point coordinates from the memory 111 to execute the positioning method of the present application, and send the obtained actual coordinates of the target point to the memory 111 for processing. storage.

The processing device 100 in this embodiment may be configured to perform each step in each method provided by the embodiment of this application. The implementation process of the positioning method is described in detail below through several embodiments.

Please refer to Figure 2, which is a flow chart of a positioning method provided by an embodiment of the present application. The specific process shown in Figure 1 will be elaborated below.

Step 201: Calculate the first coordinate of the target point of the target device through a visual algorithm.

The visual algorithm here refers to an algorithm in which the image acquisition device collects images of the target device, extracts feature points based on the image of the target device, obtains the coordinates of the feature points, and calculates the coordinates of the target point based on the coordinates of the feature points. For example, the vision algorithm may include SIFT algorithm, SURF algorithm, ORB algorithm, etc.

The target device here is a device that can be used to place objects. The target device can be a loading platform, a loading trolley, a robot, an automatic navigation device, etc. The target point is one or more points on the target device for the loading area. The first coordinate may be the coordinate of the target point on the first plane in the calibration coordinate system. The first plane in the calibration coordinate system here can be the plane where the X-axis and Y-axis in the calibration coordinate system are located, as shown in Figure 3. If the coordinate system in Figure 3 is the calibration coordinate system, the first plane can be X axis and the horizontal plane where the Y-axis is located. The first plane in the calibration coordinate system here can also be a plane parallel to the Z-axis in the calibration coordinate system. If the coordinate system in Figure 3 is a calibration coordinate system, the first plane can be the vertical plane where the Z-axis is located. It can be understood that the first plane in the above-mentioned calibration coordinate system can be selected according to actual needs.

Step 202: Calculate the second coordinate of the target point of the target device through a point laser algorithm.

The point laser algorithm here is to obtain part of the coordinate information of the reference point and part of the coordinate information of the target point through point laser, and obtain the coordinates of the target point on the second plane by calculating part of the coordinate information of the reference point. an algorithm. The second coordinate is the coordinate of the target point on the second plane in the calibration coordinate system. The reference point is one or more points in the target device for the loading area. The second plane in the calibration coordinate system here can be the plane where the X-axis and Y-axis in the calibration coordinate system are located, as shown in Figure 3. If the coordinate system in Figure 3 is the calibration coordinate system, the second plane can be X axis and the horizontal plane where the Y-axis is located. The second plane in the calibration coordinate system here can also be a plane parallel to the Z-axis in the calibration coordinate system. If the coordinate system in Figure 3 is a calibration coordinate system, the second plane can be the vertical plane where the Z-axis is located. It can be understood that the second plane in the above-mentioned calibration coordinate system can be selected according to actual needs.

Step 203: Determine the target coordinates of the target point of the target device based on the first coordinates and the second coordinates.

After the first coordinate and the second coordinate are obtained after the above calculation, the first coordinate and the second coordinate can be directly combined to obtain the target coordinate of the target point of the target device. For example, after the above calculation, the first coordinate is (X1, Y1, Rz1) and the second coordinate is (Z1, Rx1, Ry1). Then the target coordinate is the combination of the first coordinate and the second coordinate, that is, (X1, Y1 , Z1, Rx1, Ry1, Rz1).

In the above implementation process, the coordinates of the target point on the first plane in the calibration coordinate system are calculated through the visual algorithm, and then the coordinates of the target point on the first plane in the calibration coordinate system are calculated through the point laser algorithm. They can be used respectively. Accurately calculate each coordinate value of the target point to obtain the precise target coordinates of the target point. In addition, when performing visual algorithms, the real-time image data of the target device is obtained through the image acquisition device for processing. When processing through point laser, partial coordinates of the reference point are directly obtained through the laser device for processing. Since both laser equipment and image acquisition equipment can be flexibly installed and applied, and are suitable for various scenarios, the visual algorithm and the point laser algorithm are relatively flexible positioning methods. The visual algorithm and the point laser algorithm increase the use of positioning methods. flexibility.

In a possible implementation, as shown in Figure 4, step 201 includes: steps 2011-2013.

Step 2011: Match the collected image of the target device with the image template.

When matching the image of the target device with the image template, it is necessary to match the mark points on the image of the target device with the mark points on the image template. Therefore, before matching the image of the target device with the image template, it is necessary to set a marker point on the target device, and take a photo using the characteristics of the marker point to create an image template. After the execution device reaches the default position, it needs to take a photo of the marker point to obtain the actual image of the marker point. By matching the actual image of the marker point with the image template, a successful match indicates that there is a presence in the collected image of the target device. Mark points. The marking point here is the point used for marking on the target device. The actual coordinates of the marker point are the coordinates of the marker point on the image of the target device. It can be understood that the marking point may be a point specially used for marking set at a set position on the target device, or the marking point may be a fixed point selected in the load area on the target device. There is a fixed positional relationship between the mark point and the point in the load area on the target device.

The image template here is an image of the target device calibrated by the coordinate system. The image of the target device is an image of the target device taken by the execution device at a set position before executing the fetching task, and the image of the target device is a real-time image of the target device. The image of the target device can be acquired through the acquisition device on the execution device, and the image of the target device can also be acquired through the acquisition device arranged at a set position.

Based on the above embodiment, the matching method may be: matching the image of the target device with the image template by sliding the image template on the image of the target device.

Step 2012, if the matching is successful, the marker point is processed through the feature matching algorithm to output the actual coordinates of the marker point.

The feature matching algorithm here can be a SURF feature matching algorithm. The SURF feature matching algorithm is an algorithm for extracting a feature point. In order to achieve the invariance of direction rotation and scale transformation, the output result of the SURF feature matching algorithm can include the location of the feature point. information and directional information. Among them, the position information includes the position of the feature point on the image, Including the coordinate value of the first coordinate axis and the coordinate value of the second coordinate axis. The direction information includes the angle of rotation of the feature point on the third coordinate axis.

Additionally, the target device's image may fail to match the image template. If the matching fails, an alarm will be issued for the matching failure information.

Step 2013: Calculate the first coordinates of the target point of the target device based on the actual coordinates of the marker point.

The first coordinates here may include the coordinate value of the first coordinate axis of the target point, the coordinate value of the second coordinate axis, and the angle of rotation of the target point on the third coordinate axis (ie, the first rotation angle).

It can be understood that since the positions of the marker point and the target point on the target device are relatively fixed, the first coordinates of the target point can be calculated based on the actual coordinates of the marker point and the fixed value between the target point and the marker point. For example, if the actual coordinates of the marker point are calculated to be (X1, Y1, Rz1), the actual coordinates of the marker point and the fixed value between the target point and the marker point are (X, Y, Rz), then the fixed value will be compensated From the actual coordinates of the mark point, the first coordinates (X2, X2, Rz2) of the target point can be obtained.

It can be understood that the first coordinate of the target point can also be calculated by calculating the coordinate difference of the marker point. For example, if the actual coordinates of the marker point are calculated to be (X3, X3, Rz3) and the template of the marker point is (X4, Y4, Rz4), then the coordinate difference between the actual coordinates of the marker point and the template coordinates is ( ΔX, ΔY, ΔRz), and the first coordinate (X5, Y5, Rz5) of the target point can be obtained by compensating the coordinate difference to the template coordinate of the target point.

In a possible implementation, step 2013 includes: calculating a coordinate difference based on the actual coordinates of the marker point and the coordinates of the marker point template, and determining the first coordinate of the target point of the target device based on the coordinate difference.

The coordinates of the marker point template here are the coordinates of the marker point on the image template.

Understandably, the coordinate difference can be obtained directly by subtracting the actual coordinates of the marker point from the template coordinates of the marker point. The coordinate difference can also be obtained by performing difference processing on the actual coordinates of the marker point and the template coordinates of the marker point. The coordinate difference can also be obtained by combining the above two methods.

In the above implementation process, the coordinate difference is obtained by calculating the actual coordinates of the marker point and the coordinates of the marker point template. Since the positions of the marker point and the target point are relatively fixed, the coordinate difference of the marker point can also be directly used as the target. The coordinate difference of the point directly calculates the actual coordinate of the target point. The calculation of the actual coordinates of the target point is based on the marker The coordinates and coordinate differences of the points are calculated and processed based on the coordinates. They are not restricted by usage scenarios, equipment, etc., which increases the flexibility of this positioning method.

In a possible implementation, obtaining the first coordinate of the target point of the target device through the coordinate difference may include: compensating the coordinate difference to the template coordinates of the target point of the target device to obtain the target point of the target device the first coordinate.

The template coordinates of the target point here are the coordinates of the target point in the image template.

In the above implementation process, by compensating the coordinate difference to the template coordinates of the target point, the deviation between the template coordinates and the actual coordinates of the target point is eliminated, and a more accurate actual coordinate of the target point is obtained, which improves The accuracy of the first coordinate.

In a possible implementation, before step 202, the positioning method may also include: obtaining multiple reference points of the target device; calculating the first coordinate axis values of the multiple reference points and the values of the reference points through a visual algorithm. Second axis value.

The reference point here is the point in the target device. The reference point can be obtained by the target device. The reference point can be obtained by the execution device. The reference point can also be obtained by the processing device. The reference point can also be externally input to the target device, processing device or execution device. get.

First, the first coordinate axis value and the second coordinate axis value of multiple reference points are calculated through a visual algorithm. Specifically, the coordinate difference is calculated based on the actual coordinates of the marker point and the coordinates of the marker point template, and the coordinate difference is compensated to multiple The template coordinates of the reference point are used to obtain first coordinate axis values and second coordinate axis values of multiple reference points.

In a possible implementation, as shown in Figure 5, step 202 includes: steps 2021-2022.

Step 2021: Obtain the third coordinate axis values of multiple reference points and the third coordinate axis value of the target point through a point laser.

Based on the above embodiments, the point laser can obtain the third coordinate axis value of the reference point, and the point laser can also obtain the third coordinate axis value of the target point.

Step 2022: Obtain the second coordinates of the target point of the target device based on the reference point coordinates of the multiple reference points and the third coordinate axis value of the target point.

The reference point coordinates of the reference point here include the first coordinate axis value of the reference point, the second coordinate axis value of the reference point, and the third coordinate axis value of the reference point.

In the above embodiment, the first coordinate axis value, the second coordinate axis value and the third coordinate axis value of multiple reference points are obtained. The second rotation angle and the third rotation angle of the target point can be calculated based on the first coordinate axis value, the second coordinate axis value and the third coordinate axis value of the plurality of reference points. And since the above-mentioned point laser acquires the third coordinate axis value of the target point, the second coordinate of the target point can be obtained based on the second rotation angle, the third rotation angle and the third coordinate axis value.

Since the vision algorithm can only calculate the first coordinate axis value and the second coordinate axis value of the reference point, in order to obtain the complete coordinates of the reference point, it is necessary to obtain the third coordinates of the target point and multiple reference points through a point laser. axis value. Through the above-mentioned point laser and vision algorithm, the first coordinate axis value, the second coordinate axis value, the third coordinate axis value of multiple reference points and the third coordinate axis value of the target point can be jointly obtained.

In the above implementation process, after the actual coordinates of the marker point are calculated based on the vision algorithm, the first coordinates of the reference point can also be calculated based on the vision algorithm, just like the algorithm for the first coordinates of the target point. Since the first coordinate does not contain the third coordinate axis value, the third coordinate axis value of the target point and the reference point is obtained through the point laser to obtain the values of each coordinate axis of the reference point. The point laser algorithm is used to compensate The shortcomings of the visual algorithm obtain values that cannot be obtained through the visual algorithm, which is conducive to further calculation of the actual coordinates of the target point, simplifying the calculation difficulty, and improving the efficiency and accuracy of calculating the actual coordinates of the target point.

In a possible implementation, step 2021 includes: obtaining a first reference point, a second reference point, and a third reference point.

It can be understood that the above-mentioned first reference point, second reference point and third reference point can be implemented through an external input execution device, processing device or target device. The above-mentioned first reference point, second reference point and third reference point can be obtained through the target device according to the set rules. The above-mentioned first reference point, second reference point and third reference point can also be obtained through the execution device according to the set rules. The above-mentioned first reference point, second reference point and third reference point can also be obtained through the processing device according to the set rules.

Wherein, the first reference point and the second reference point are at the same position on the first coordinate axis, and the first reference point and the third reference point are at the same position on the second coordinate axis. For example, if the X-axis of the first reference point and the second reference point are the same, then the Y-axis of the first reference point and the third reference point are the same. If the Y-axis of the first reference point and the second reference point are the same, then the X-axis of the first reference point and the third reference point are the same. As shown in Figure 3, if the coordinate system shown in Figure 3 is the coordinate system where the target device is located. Then point B shown in the figure is the first reference point, point A is the second reference point, and point C is the third reference point.

In a possible implementation, step 2023 includes: calculating the first included angle based on the coordinates of the first reference point and the coordinates of the second reference point, and calculating the second included angle based on the coordinates of the first reference point and the third reference point. angle, and obtain the second coordinates of the target point of the target device according to the first included angle, the second included angle and the coordinates of the multiple reference points.

The first included angle is the included angle between the line connecting the first reference point and the second reference point and the second coordinate axis. The second included angle is the included angle between the line connecting the first reference point and the third reference point and the first coordinate axis. For example, if the X-axis of the first reference point and the second reference point are the same, the Y-axis of the first reference point and the third reference point are the same. Then the first included angle is the angle between the line connecting the first reference point and the second reference point and the Y-axis, and the second included angle is the angle between the line connecting the first reference point and the third reference point and the X-axis. angle. If the Y-axis of the first reference point and the second reference point are the same, the X-axis of the first reference point and the third reference point are the same. Then the first included angle is the angle between the line connecting the first reference point and the second reference point and the X-axis, and the second included angle is the angle between the line connecting the first reference point and the third reference point and the Y-axis. angle.

The first included angle corresponds to the second rotation angle of the target point, and the second included angle corresponds to the third rotation angle of the target point. For example, if the first included angle is the angle between the line connecting the first reference point and the second reference point and the Y-axis, the second included angle is the angle between the line connecting the first reference point and the third reference point and the X-axis. the angle between. Then the first included angle corresponds to the Rx of the target point, and the second included angle corresponds to the Ry of the target point. If the first included angle is the angle between the line connecting the first reference point and the second reference point and the X-axis, the second included angle is the angle between the line connecting the first reference point and the third reference point and the Y-axis. angle. Then the first included angle corresponds to Ry of the target point, and the second included angle corresponds to Rx of the target point.

Based on the same application concept, the embodiment of the present application also provides a positioning device corresponding to the positioning method. Since the principle of solving the problem of the device in the embodiment of the present application is similar to that of the aforementioned positioning method embodiment, the device in this embodiment is For implementation, please refer to the description in the embodiments of the above method, and repeated details will not be described again.

Please refer to FIG. 6 , which is a schematic diagram of the functional modules of the positioning device provided by the embodiment of the present application. Each module in the positioning device in this embodiment is configured to perform each step in the above method embodiment. The positioning device may include a first calculation module 301, a second calculation module 302, and a processing module 303; wherein,

The first calculation module 301 is configured to calculate the first coordinates of the target point of the target device through a visual algorithm, where the first coordinates are the coordinates of the target point on the first plane in the calibration coordinate system.

The second calculation module 302 is configured to calculate the second coordinates of the target point of the target device through a point laser algorithm, where the second coordinates are the coordinates of the target point on the second plane in the calibration coordinate system.

The processing module 303 is configured to obtain the target coordinates of the target point of the target device according to the first coordinates and the second coordinates.

In a possible implementation, the first calculation module 301 may also be configured to: match the collected image of the target device with an image template, where the image template is the target device calibrated by the coordinate system. image; if the match is successful, the marker point is processed through the feature matching algorithm to output the actual coordinates of the marker point, the marker point is the set marker point position on the target device, and the actual coordinates of the marker point are the marker point coordinates on the image of the target device; and calculate the first coordinates of the target point of the target device based on the actual coordinates of the marker point.

In a possible implementation, the first calculation module 301 may be configured to: calculate a coordinate difference based on the actual coordinates of the marker point and the template coordinates of the marker point, where the template coordinates of the marker point are the markers. The coordinates of the point on the image template; the first coordinate of the target point of the target device is obtained through the coordinate difference.

In a possible implementation, the first calculation module 301 may be configured to: obtain multiple reference points of the target device; calculate the sum of the first coordinate axis values of the multiple reference points through a visual algorithm. The second coordinate axis value of the reference point.

In a possible implementation, the first calculation module 301 may be configured to: compensate the coordinate difference to the template coordinates of the target point of the target device to obtain the first coordinate of the target point of the target device. . In a possible implementation, the second calculation module 302 may also be configured to: obtain the third coordinate axis values of a plurality of reference points and the third coordinate axis values of the target point through a point laser; Obtain the second coordinate of the target point of the target device according to the reference point coordinates of a plurality of the reference points and the third coordinate axis value of the target point. The reference point coordinates of the reference point include the reference point. The first coordinate axis value of the reference point, the second coordinate axis value of the reference point, and the third coordinate axis value of the reference point.

In a possible implementation, the second calculation module 302 may be configured to: obtain a first reference point, a second reference point, and a third reference point. The first reference point and the second reference point are The positions of the points on the first coordinate axis are the same, and the positions of the first reference point and the third reference point on the second coordinate axis are the same.

In a possible implementation, the second calculation module 302 may be configured to: calculate the first included angle according to the coordinates of the first reference point and the coordinates of the second reference point. The angle is the included angle between the line connecting the first reference point and the second reference point and the second coordinate axis; the second included angle is calculated according to the coordinates of the first reference point and the third reference point, and the The second included angle is the included angle between the line connecting the first reference point and the third reference point and the first coordinate axis; according to the first included angle, the second included angle and the third included angle of the target point The three-coordinate axis value obtains the second coordinate of the target point of the target device.

In addition, embodiments of the present application can also provide a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the positioning method described in the above-mentioned method embodiments are executed. .

The computer program product of the positioning method provided by the embodiment of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the positioning method described in the above method embodiment. For details, please refer to the above method embodiments and will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and functions of the devices, methods and computer program products according to multiple embodiments of the present application. operate. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more components for implementing the specified logical function(s). Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts. , or can be implemented using a combination of specialized hardware and computer instructions.

In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, each module can exist alone, or two or more modules can be integrated to form an independent part.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the relevant technology or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several The instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. . It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

The above descriptions are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the protection scope of this application. It should be noted that similar reference numerals and letters represent similar items in the following figures, therefore, once an item is defined in one figure, it does not need further definition and explanation in subsequent figures.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application. should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Industrial applicability

This application provides a positioning method, device, electronic equipment and readable storage medium, which calculates the first coordinate of the target point of the target device through a visual algorithm; calculates the second coordinate of the target point of the target device through a point laser algorithm; The target coordinates of the target point of the target device are obtained according to the first coordinates and the second coordinates. The embodiment of the present application calculates the target coordinates of the target point of the target device through two coordinate calculation methods: visual algorithm and laser point algorithm. The actual coordinates of the target point of the target device can be calculated only through images and some calculation rules. This makes the calculation of the target point coordinates of the target device's target point more flexible.

Furthermore, it will be appreciated that the positioning method and device of the present application are reproducible and can be used in a variety of industrial applications. For example, the positioning method and device of the present application can be used in any application that requires increased positioning flexibility.

Claims

A positioning method, wherein the positioning method includes:

Calculate the first coordinates of the target point of the target device through a visual algorithm, where the first coordinates are the coordinates of the target point on the first plane in the calibration coordinate system;

Calculate the second coordinates of the target point of the target device through a point laser algorithm, where the second coordinates are the coordinates of the target point on the second plane in the calibration coordinate system;

The target coordinates of the target point of the target device are determined according to the first coordinates and the second coordinates.
The method according to claim 1, wherein calculating the first coordinate of the target point of the target device through a visual algorithm includes:

Match the collected image of the target device with an image template, where the image template is an image of the target device whose coordinate system has been calibrated;

If the match is successful, the marker point is processed through the feature matching algorithm to output the actual coordinates of the marker point. The marker point is the set marker point position on the target device. The actual coordinates of the marker point are the marker point location. The coordinates on the image of the target device;

Calculate the first coordinates of the target point of the target device according to the actual coordinates of the marker point.
The method according to claim 2, wherein calculating the first coordinates of the target device according to the actual coordinates of the marker point includes:

The coordinate difference is calculated based on the actual coordinates of the marker point and the template coordinates of the marker point, where the template coordinates of the marker point are the coordinates of the marker point on the image template;

The first coordinate of the target point of the target device is determined based on the coordinate difference.
The method of claim 3, wherein determining the first coordinate of the target point of the target device through the coordinate difference includes: compensating the coordinate difference to the target point of the target device. The template coordinates of the target point obtain the first coordinates of the target point of the target device, and the template coordinates of the target point are the coordinates of the target point in the image template.
The method according to any one of claims 1 to 4, wherein before calculating the second coordinates of the target point of the target device through a point laser algorithm, the method further includes:

Obtain multiple reference points of the target device;

The first coordinate axis values of a plurality of reference points and the second coordinate axis values of the reference points are calculated through a visual algorithm.
The method according to claim 5, wherein the obtaining the second coordinates of the target point of the target device through a point laser algorithm includes:

Acquire the third coordinate axis values of a plurality of the reference points and the third coordinate axis values of the target points through a point laser;

The second coordinates of the target point of the target device are obtained according to the reference point coordinates of a plurality of the reference points and the third coordinate axis value of the target point. The reference point of the reference point is The coordinates include a first coordinate axis value of the reference point, a second coordinate axis value of the reference point, and a third coordinate axis value of the reference point.
The method according to claim 6, wherein the obtaining multiple reference points of the target device includes:

Obtain the first reference point, the second reference point and the third reference point. The first reference point and the second reference point are at the same position on the first coordinate axis. The first reference point and the third reference point are The points are at the same position on the second coordinate axis.
The method according to claim 7, wherein the second coordinate of the target point of the target device is obtained based on a plurality of the reference point coordinates and a third coordinate axis value of the target point, include:

A first included angle is calculated based on the coordinates of the first reference point and the coordinates of the second reference point. The first included angle is the line connecting the first reference point and the second reference point and the third reference point. The angle between the two coordinate axes;

A second included angle is calculated based on the coordinates of the first reference point and the third reference point. The second included angle is the line connecting the first reference point and the third reference point and the third reference point. The angle between a coordinate axis;

The second coordinates of the target point of the target device are obtained according to the first included angle, the second included angle and the third coordinate axis value of the target point.
A positioning device, wherein the positioning device includes:

A first calculation module, the first calculation module is configured to calculate the first coordinate of the target point of the target device through a visual algorithm, where the first coordinate is the first coordinate of the target point in the calibration coordinate system. Coordinates on the plane;

A second calculation module, the second calculation module is configured to calculate the second coordinates of the target point of the target device through a point laser algorithm, the second coordinates being the target point at the Coordinates on the second plane in the calibration coordinate system;

and a processing module configured to obtain the target coordinates of the target point of the target device according to the first coordinates and the second coordinates.
An electronic device, wherein the electronic device includes: a processor and a memory, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are When the processor is executed, the steps of the method according to any one of claims 1 to 8 are performed.
A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the method according to any one of claims 1 to 8 when run by a processor.