KR20220058846A - Robot positioning method and apparatus, apparatus, storage medium - Google Patents

Abstract

The present invention relates to a robot positioning method and apparatus, apparatus and storage medium, the method comprising the steps of: acquiring first position information of the robot at a first time; acquiring a second image of the robot at a second time, and obtaining first expected position information of the robot at a second time based on the second image; collecting motion parameters of the robot in the process from the first time to the second time; calculating and obtaining second expected position information of the robot at a second time based on the first position information and the motion parameter; and obtaining second position information of the robot at a second time through the first expected position information and the second expected position information.

Description

Robot positioning method and apparatus, apparatus, storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

The present invention is filed based on a Chinese patent application with an application number of 202011157118.7, an application date of October 26, 2020, entitled "Robot Positioning Method and Apparatus, Apparatus, and Storage Medium", and priority of the Chinese patent application All contents of the above Chinese patent application are incorporated herein by reference.

The present invention relates to the field of artificial intelligence technology, and more particularly, to a robot positioning method and apparatus, an apparatus, and a storage medium.

In recent years, artificial intelligence has become more and more popular, includes multiple functions, and the application of integrated positioning systems in various fields is becoming more and more common. For example, in artificial intelligence (AI) education, both parties in the class often encounter content such as positioning, planning, control, and multi-agent, and the most basic and important part is the robot positioning system.

The present invention provides a robot positioning method and apparatus, an apparatus, and a storage medium.

The first technical solution provided by the present invention is to provide a robot positioning method, the method comprising:

acquiring first position information of the robot at a first time;

obtaining a second image of the robot at a second time, and obtaining first expected position information of the robot at a second time based on the second image;

collecting motion parameters of the robot in a process from a first time to a second time;

calculating and obtaining second expected position information of the robot at the second time based on the first position information and the motion parameter; and

and obtaining second position information of the robot at the second time through the first expected position information and the second expected position information.

In this way, by fusing the detection result of the visual system and the detection result of the dynamic system, the obtained position information is made more accurate.

In some embodiments of the present invention, obtaining the first position information of the robot at the first time comprises:

acquiring a first image of the robot at the first time; and

and acquiring the first position information of the robot at the first time based on the first image.

In some embodiments of the present invention, the first image includes a first reference, and obtaining the first position information of the robot at the first time based on the first image comprises:

obtaining the first reference in a simulation sand table;

determining a pixel position of the first reference object and a first pixel position of the robot in the first image;

determining a simulation position of the first reference object in the simulation sand table at a first time;

calculating and obtaining a first projection relationship using the pixel position of the first reference and the simulated position of the first reference in the simulation sand table; calculating and obtaining a simulation position of the robot in the simulation sand table based on the first projection relationship and a first pixel position of the robot; and

and determining the first position information of the robot at a first time by using the simulation position of the robot in the simulation sand table.

In this way, by using the visual system to determine the initial position of the robot by way of a simulation sand table, there is no need to calibrate the reference additionally, and the additional calibration time is reduced.

In some embodiments of the present invention, the second image includes a second reference, and the step of obtaining the first expected position information of the robot at the second time based on the second image comprises:

obtaining the second reference in a simulation sand table;

determining a pixel position of the second reference object and a second pixel position of the robot in the second image;

determining a simulation position of the second reference object in the simulation sand table at a second time;

calculating and obtaining a second projection relationship using the pixel position of the second reference object and the simulated position of the second reference object in the simulation sand table;

calculating and obtaining a simulation position of the robot in the simulation sand table based on the second projection relationship and a second pixel position of the robot; and

and determining first expected position information of the robot at a second time by using the simulation position of the robot in the simulation sand table.

In this way, by using the visual system to determine the end-point position of the robot through the manner of a simulation sand table, there is no need to calibrate the reference additionally, and the additional calibration time is reduced.

In some embodiments of the present invention, determining the pixel position of the first reference object and the first pixel position of the robot in the first image comprises:

performing identification on the first image using a first deep learning network to determine a pixel position of the first reference object and a first pixel position of the robot in the first image; The determining of the pixel position of the second reference object and the second pixel position of the robot in the second image includes performing identification on the second image using a first deep learning network, determining the pixel position of the second reference object and the second pixel position of the robot, wherein the first deep learning network is one of RCNN deep network structure, SSD deep network structure, Yolo deep network structure, RetinaNet network structure or any combination.

In some embodiments of the present invention, the step of obtaining the first image of the robot at the first time may further include obtaining the direction of the robot at the first time based on the first image. include

Here, in the step of obtaining the direction of the robot at the first time based on the first image, the identification of the area image in which the robot is located is performed using a second deep learning network, determining a direction, wherein the second deep learning network includes one or any combination of a ResNet deep network architecture, a MobileNet deep network architecture, a GhostNet deep network architecture, and an EfficientNet deep network.

In this way, by detecting the direction of the robot, it makes the result obtained more accurate when the position information of the robot is subsequently calculated through the dynamics system.

In some embodiments of the present invention, the positioning method comprises:

Further comprising the step of obtaining the historical motion parameter of the robot at the first time,

Collecting the movement parameters of the robot in the process from the first time to the second time comprises:

acquiring a current motion parameter of the robot at the second time; and

and acquiring the motion parameter of the robot in the process from a first time to a second time through the historical motion parameter and the current motion parameter.

In this way, by calculating the movement distance of the robot in the process from the first time to the second time, and combining it with the direction of the robot, when the position information of the robot is subsequently calculated through the dynamics system, the result obtained make it more accurate.

In some embodiments of the present invention, the step of calculating and obtaining the second expected position information of the robot at the second time based on the first position information and the motion parameter comprises:

Through the movement parameter of the robot in the process from the first time to the second time, it is combined with the direction of the robot in the first time to obtain second expected position information of the robot at the second time including the steps of

In this way, by using the dynamic system to detect the position information of the robot at the second time, the hardware cost is reduced.

In some embodiments of the present invention, the step of obtaining the second position information of the robot at the second time through the first expected position information and the second expected position information comprises:

and performing a weighted average on the first expected position information and the second expected position information using a Kalman filtering method to obtain second position information of the robot at the second time. .

In this way, fusion is performed on the position information obtained by the visual system and the position information obtained by the dynamics system to obtain the position information of the robot at the second time, thereby improving the positioning accuracy and improving the ease of deployment of the system. increase

The second technical solution provided by the present invention is to provide a robot positioning device, the device comprising:

a first position acquiring unit configured to acquire first position information of the robot at a first time, and first expected position information of the robot at a second time;

a parameter acquisition unit configured to collect movement parameters of the robot in a process from a first time to a second time;

a second position acquiring unit configured to calculate and obtain second expected position information of the robot at the second time based on the first position information and the motion parameter;

and a calibration unit configured to obtain second position information of the robot at the second time through the first expected position information and the second expected position information.

The third technical solution provided by the present invention is a robot positioning device including a memory and a processor, wherein a program instruction is stored in the memory, and the processor calls the program instruction from the memory to execute the robot positioning method .

A fourth technical solution provided by the present invention is a computer-readable storage medium, a memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute the robot positioning method. do.

A fifth technical solution provided by the present invention is a computer program including a computer readable code, and when the computer readable code is operated in a robot positioning device and executed by a process of the robot positioning device, the robot positioning method run

The robot positioning method provided in the present invention is a method of obtaining position information of the robot by fusing the robot positioning result obtained through two different methods, that is, the detection result of the visual system and the detection result of the dynamic system, and the positioning method makes the acquired position information more accurate, and effectively improves the positioning accuracy of the robot.

In order to more clearly explain the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings that will be used in the description of the embodiments, and the drawings in the description below are only some embodiments of the present invention, It is apparent that those skilled in the art can obtain other drawings according to these drawings, even under the premise that creative labor is not given.
1 is a flowchart illustrating an embodiment of a robot positioning method according to an embodiment of the present invention.
2 is a flowchart illustrating an embodiment of step S11 of FIG. 1 according to an embodiment of the present invention.
3 is a flowchart illustrating an embodiment of step S12 of FIG. 1 according to an embodiment of the present invention.
4 is a flowchart illustrating an embodiment of step S13 of FIG. 1 according to an embodiment of the present invention.
5 is a diagram illustrating the principle of a robot positioning method according to an embodiment of the present invention.
6 is a structural illustration of an embodiment of a robot positioning device according to an embodiment of the present invention.
7 is a structural illustration of an embodiment of a robot positioning device according to an embodiment of the present invention.
8 is a structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the embodiments of the present invention, and it is clear that the described embodiments are only some of the embodiments of the present invention and not all embodiments. Based on the embodiment of the present invention, other embodiments obtained by a person skilled in the art without creative labor all fall within the protection scope of the present invention.

The terms “first”, “second”, and “third” of the present invention are for illustrative purposes only, and are understood to indicate or imply relative importance, or to implicitly indicate the number of technical features to be represented. you must not do As will be appreciated from this, features defined as “first”, “second”, “third” may explicitly or implicitly include at least one of said features. In the description of the present invention, "a plurality" means at least two, for example, two, three, and the like, unless otherwise defined. All directional indications (eg, up, down, left, right, front, back...) among the embodiments of the present invention are relative positional relationship between each member in a specific posture (as shown in the drawing), operation situation It is only for interpreting the etc., and when the specific posture is changed, the direction indication is also changed correspondingly. Also, the terms “comprising” and “having” and any variations thereof are intended to include non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to already listed steps or units, but may optionally further include, or optionally include, non-listed steps or units; It further includes other steps or units specific to the method, product, or device.

In the present specification, “embodiment” means that a specific feature, structure, or characteristic described in combination with an embodiment may be included in at least one embodiment of the present invention. The appearances of the phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are they mutually exclusive or alternative embodiments of other embodiments. A person skilled in the art clearly and implicitly understands that the embodiments described herein may be combined with other embodiments.

Due to the variety of application scenarios, it is necessary to consider the difficulty of positioning system deployment, and spending a lot of time on environmental modeling often degrades the user experience, wastes a lot of time, and also requires a lot of computational resources for the positioning system, Often it lacks accuracy and is expensive. For example, schools use a lot of robots for educational purposes. If a high-performance computing device is embedded inside the robot to support various positioning algorithms, the cost of the robot increases significantly, and it is difficult to manufacture a high-performance implantable positioning device at low cost. it's difficult. Therefore, we use limited hardware resources to achieve a better positioning effect and obtain a high-precision, low-cost, easy-to-deploy robot positioning system.

An embodiment of the present invention provides a robot positioning method and apparatus, an apparatus, and a storage medium. Collecting movement parameters of the robot in the process from the first time to the second time by acquiring the first position information of the robot at the first time and the first expected position information at the second time, and , calculating and obtaining second expected position information of the robot at a second time based on the first position information and the motion parameter; The second position information of the robot at the second time is obtained through the first expected position information and the second expected position information. An embodiment of the present invention performs fusion through a combination of a visual positioning system and a dynamic positioning system to obtain a robot positioning result with relatively high accuracy.

Here, the visual positioning system performs positioning based on the image acquired by the camera, and the dynamic positioning system performs positioning based on the initial position of the robot and the working parameters of the robot. When the visual positioning system performs positioning, obtain a second image of the robot at the second time through the camera, and combine it with the simulation sand table based on the second image, so that the first prediction of the robot in the simulation sand table is obtained. determine location information; When the dynamic positioning system performs positioning, it acquires the robot's movement parameters and the robot's direction in the process from the first time to the second time, that is, the robot moves in a specific direction through the robot's direction and movement parameters. The distance can be determined, and when combined with the first position information of the robot at the first time, the second expected position information of the robot at the second time is obtained. Here, in order to accurately determine the positioning position of the robot, the first expected position information obtained by the visual positioning system through Kalman filtering and the second expected position information obtained by the dynamic positioning system are fused, that is, by performing a weighted average, finally to acquire the second position information of the robot at an accurate second time.

The execution subject of the robot positioning method may be a robot positioning device, for example, the robot positioning method may be executed by a terminal device or a server or other processing device, where the terminal device is a user equipment (UE), a mobile device It may be a device, a user terminal, a terminal, a cellular phone, a wireless telephone, a personal digital assistant (PDA), a hand-held device, a computing device, an in-vehicle device, a wearable device, and the like. In some possible implementation manners, the robot positioning method may be implemented through a manner in which a processor calls computer readable instructions stored in a memory.

The present invention will be described in detail in conjunction with the drawings and examples below.

Referring to FIG. 1 , it is a flowchart illustrating an embodiment of a robot positioning method according to an embodiment of the present invention, and the method includes the following steps.

In step S11, first position information of the robot at the first time is acquired.

In one embodiment of the embodiment of the present invention, when performing robot positioning, the first position information of the robot at the first time and the first expected position information of the robot at the second time are obtained by using the positioning system directly. And, the positioning system may be a positioning system of a global positioning system (Global Positioning, GPS).

In one embodiment of the embodiment of the present invention, it is also possible to acquire a first image of the robot at a first time through an imaging device such as a camera. Here, the robot is a movable mechanical device or smart device, for example, a forklift, a mechanical trolley, or the like.

In one possible embodiment, a timer may be integrated in the camera, and when the set time of the timer reaches the first time, the camera takes a picture of the robot to obtain a first image. Here, the position of the camera may or may not be fixed; The shooting angle of the camera may or may not be fixed; Here, the camera may be installed at a specified position and not rotated, that is, the shooting range may be fixed, and the camera may also be installed and rotated at the specified position, that is, the reflection range may change; Of course, the camera can also be installed on a mobile device. The present invention does not limit the position and shooting range of the camera as long as the camera can take a picture of the robot.

In one possible embodiment, the first image acquired by the camera may be uploaded to the robot positioning device, for example, the robot positioning device and the camera may be communicatively connected, and after the camera acquires the first image, the communication connection is established to send the first image to the robot positioning device via As long as it can be obtained, it is not specifically limited thereto.

Here, when the first image is uploaded to the robot positioning device, the robot positioning device uses the visual system to acquire the first position information of the robot at the first time based on the first image, and specifically, continuously See FIG. 1 .

The robot positioning device acquires the first position information of the robot at the first time based on the first image by using the visual system, and in one possible embodiment, combines it with the first image through the method of scenario modeling, determines the first location information of , and here, referring to FIG. 2 , it is a flowchart illustrating an embodiment of step S11 of FIG. 1 according to an embodiment of the present invention, and specifically includes the following steps.

In step S111, a first reference in the simulation sand table is obtained.

In one embodiment of the embodiment of the present invention, the first position information of the robot is determined through a method of building a simulation sand table. It should be understood that the constructed simulation sand table is a simulation blueprint of the space where the robot is located.

Here, the simulation sand table is built according to the scenario where the robot is located, and the simulation sand table can be built according to the scenario where the robot is located in a 1:1 ratio, for example, the size of all objects in the simulation sand table. , position, coordinates and the size, position, and coordinates of all objects in the scenario where the robot is located are the same. Of course, the simulation sand table can also be built according to a 1:2 ratio, for example, the size, position, and coordinates of all objects in the simulation sand table are the size, position, and coordinates of all objects in the scenario where the robot is located. It is twice or 1/2 times the coordinates, of course, it is also possible to build the simulation sand table according to other ratios, and the present invention is not limited thereto.

In one possible embodiment, in consideration of the simplicity of calculation, a simulation sand table may be built using a ratio of 1:1, and there is no need to perform ratio conversion in a subsequent calculation process, thereby reducing the amount of calculation. In another embodiment, taking into account the space occupied by the simulation sand table, for example, when the scenario where the robot is positioned is too large, the simulation sand table using a ratio of 1:2, 1:3, 1:4, etc. By building this, it is possible to reduce the space occupied by the simulation sand table. For example, all of the above processes may be performed in a robot positioning device, a simulation sand table is built in the robot positioning device, and when the robot positioning device acquires the first image from the camera, the robot positioning device uses a visual system to determine the first position information of the robot based on the simulation sand table and the first image.

Here, when positioning is performed on the robot using the visual system, there must be one first reference object in the first image and the simulation sand table at the same time, and the first reference object is the first image projected on the simulation sand table can be used to calculate the first projection relation. Here, the first reference can be selected according to the first image, for example, other than the robot, the first image further includes the same table as in the simulation sand table, so that the table can be used as the first reference, Also for example, in addition to the robot, the first image further includes the same TV as in the simulation sand table, so that the TV can be used as the first reference, that is, the first reference is shared with the first image in the simulation sand table object that becomes

In step S112, a pixel position of a first reference object in the first image and a first pixel position of the robot are determined.

In an embodiment, identification may be performed on the first image using the first deep learning network to determine the pixel position of the first reference in the first image, and the first pixel position of the robot.

In one embodiment, the first deep learning network may be a model integrating a deep learning network with a positioning function, and when a first image is input to the model, the model performs identification on the first image, Obtain a pixel position of the first reference in the first image and a first pixel position of the robot in the first image.

In an embodiment of one embodiment, detecting the first image using the first deep learning network to determine the pixel position of the first reference in the first image and the first pixel position of the robot in the first image; The first deep learning network includes a target detection (Region-CNN, RCNN) deep network structure, a target detection (Single Shot MultiBox Detector, SSD) deep network structure, a target detection (You Only Look Once, Yolo) deep network structure, and a RetinaNet network structure. and the like.

In an embodiment of one embodiment, in consideration of cost and accuracy of location coordinate detection, the location coordinate detection may be performed through the Yolo deep network structure. Here, detection may be performed using the Yolo-tiny deep network structure in the Yolo deep network structure.

In the Yolo deep network architecture, the algorithm's mindset divides the whole image into some grids, predicting some possible bounding frames in the cell grid whose center is on the objects in the grid, and by providing confidence, these algorithms can Since the result of detection frame can be obtained, the speed of the algorithm is faster than that of the two-step Faster-RCNN series algorithm. Since the shape and color of the object to be detected in the application scenario are relatively fixed, the detection according to the small network structure for such a fast algorithm has high accuracy, and at the same time, because the space occupied by the computational resources is small, the mobile terminal calculation speed is relatively slow The required real-time detection effect can be achieved in a central processing unit (CPU), for example, a Raspberry Pi, so that the required cost is lower.

Here, the pixel position of the first reference object and the first pixel position of the robot obtained in the first image may be representative points, for example, the pixel position of the central point of the first reference and the pixel of the central point of the robot in the first image obtain the position, and also obtain the pixel position of the frame of the first reference object and the pixel position of the frame of the robot, and also obtain the pixel position of the diagonal of the frame of the first reference and the pixel position of the diagonal of the frame of the robot may be selected, but is not limited thereto.

An image is constructed by overlapping a plurality of R (red sub-pixel), G (green sub-pixel), and B (blue sub-pixel) pixel points to create different colors. Therefore, the first deep learning network can be used to obtain the pixel positions in the first image of the robot and the first reference, where the robot is the first pixel position in the first image is the position of the sub-pixel in the image .

In step S113, the simulation position of the first reference object in the simulation sand table at the first time is determined.

Here, the first reference object selected from the first image and the first reference object selected from the simulation sand table are the same object, and after the simulation sand table is built, the simulation positions of all objects in the simulation sand table are already known.

In an embodiment of the present invention, when acquiring the simulated position of the first reference object, it should be determined corresponding to the pixel position of the first reference object. For example, in one embodiment, when the pixel position of the obtained first reference is the pixel position of the center point of the first reference, the simulated position of the obtained first reference is the simulated position of the center point of the first reference ; Also for example, when the pixel position of the obtained first reference object is the pixel position of the frame of the first reference object, the simulated position of the obtained first reference object is the simulation position of the frame of the first reference object.

In step S114, a first projection relationship is calculated and obtained by using the pixel position of the first reference object and the simulation position of the first reference object in the simulation sand table.

Here, after determining the pixel position of the first reference object in the first image and the simulation position of the first reference object in the simulation sand table through the above steps S112 and S113, the pixel position of the first reference object in the first image and calculating the first projection relationship using the simulation position of the first reference object in the simulation sand table.

In one embodiment, the pixel position of the first reference in the acquired first image is the center point pixel position of the first reference, and the simulation position of the first reference in the simulation sand table is the center point simulation position of the first reference Assume that Here, assuming that the center point pixel position of the first reference is (u, v, w) and the center point simulation position of the first reference is (x', y', w'), the center point pixel of the first reference A relational equation is constructed based on the position (u, v, w) and the simulated position of the center point of the first reference object.

A relational equation constructed through the pixel positions (u, v, w) of the center point of the first reference and the simulated position of the center point of the first reference is the following formula (1).

(1)

(One)

는 제1 투영 관계이다.Here, in formula (1),

is the first projection relationship.

In the embodiment of the present invention, there are three or more references to be selected, and the above formula (1) Build several sets of relational equations through the method of If the obtained results do not match, processing is performed on a plurality of projection relationships through a method such as weighted average to obtain accurate results. Here, in the same first image, the first projection relationship of different first references to the simulation sand table is the same.

In one embodiment, since the first image is a two-dimensional image, the position coordinates of the obtained first reference in the first image are also two-dimensional coordinates, and thus the pixel position of the first reference in the first image ( w of u, v, w) is a constant and is not a number indicated by the Z coordinate. Here, w may be 1. Specifically, since all objects are set with respect to the ground, the simulation sand table constructed in the present technical solution is a two-dimensional simulation sand table, and, therefore, the simulation position (x', y', w') are also two-dimensional coordinates, that is, w' is also a constant, and likewise, it is not a number indicated by the Z coordinate. Here, w' may also be 1. Therefore, the lower right corner number a_33 in the first projection relation according to formula (1) is always 1.

In the first projection relation in Equation (1), there are all 8 unknowns, the results of these 8 unknowns must be obtained, and 4 pairs of coordinates must be calculated. Here, each coordinate pair includes a pixel position in one first image and a simulation position in one simulation sand table. The four pairs of coordinates may be selected from the same first reference object, and after selecting four pixel positions located in the first image from the same first reference object, four more simulation positions located in the simulation sand table are selected. Here, the first reference may be the same first reference as the first reference used when constructing the relational equation, or may be a different first reference, and the selected four points are, when constructing the relational equation It may be the same as or different from the point selected from the first reference used in .

Here, when the value of the relational equation is obtained, in order to effectively improve the accuracy and robustness of the calculation result of the first projection relation, it is calculated using a Random Sample Consensus (RANSAC) algorithm and finally the 1 You can output the projection relationship. Here, the random sample consistency algorithm can estimate the parameters of the mathematical model through an iterative method from one observation data set including “outliers”, improving the accuracy and robustness of the calculation of the first projection relationship. can do it

An embodiment of the present invention establishes a relational equation through the pixel position of the first reference in the first image and the simulated position in the simulation sand table, and obtains a value of the relational equation to obtain a first projection relation. Here, through the calculation in combination with the random sample coincidence algorithm, the accuracy and robustness of the calculation of the first projection relationship are improved.

In step S115, the simulation position of the robot in the simulation sand table is calculated and obtained based on the first projection relationship and the first pixel position of the robot.

In an embodiment of the present invention, when the first pixel position of the robot in the first image has already been obtained in step S112, and the first projection relationship of the first image projected on the simulation sand table is already obtained in step S114, It can be obtained by calculating the simulation position of the robot in the simulation sand table through the first projection relationship and the first pixel position of the robot. Here, the simulation position of the robot in the simulation sand table is determined through the method of formula (1) above.

In step S116, first position information of the robot at the first time is determined using the simulation position of the robot in the simulation sand table.

In one embodiment, when the simulation sand table is built at a ratio of 1:1 of the real scenario, the simulation position of the robot in the simulation sand table that can be calculated and obtained is the first position information of the robot at the first time ; When the simulation sand table is built at a ratio of 1:2 of the real scenario, after calculating the simulation position of the robot in the simulation sand table, it is converted through the related ratio to obtain the first position information of the robot at the first time. get

An embodiment of the present invention builds a simulation sand table, performs the first projection relationship calculation according to using an object existing in the simulation sand table as a reference, and the method does not require additional markers to be set in the sand table Therefore, the operation is simple; Identify the pixel position of the robot in the first image through deep learning, combine with the first projection relationship to determine the simulation position of the robot in the simulation sand table, and determine the first position information of the robot at the first time do. The positioning process of these robots simplifies the operation, reduces costs, and greatly improves the user experience.

In one embodiment, when acquiring the first position information of the robot at the first time, it is necessary to detect the direction of the robot, that is, to detect the angle of the robot.

Here, the direction of the robot may be detected through the angular posture prediction model. Here, first performing identification on the first image by using the first deep learning network, and obtaining the position of the robot in the first image; Next, the area image where the robot is located is extracted, the extracted area image is input to the angle prediction model, the angle of the robot is detected through the angle prediction model, and the direction of the robot is obtained. Here, after obtaining the direction of the robot, it is possible to know the direction of movement of the robot from the first time to the second time.

Here, the second deep learning network may be integrated into the angle prediction model, and the second deep learning network is used to identify the area image where the robot is located to determine the direction of the robot. Here, the second deep learning network may be, for example, a convolutional network structure used for numerical regression in related technologies such as ResNet deep network structure, MobileNet deep network structure, GhostNet deep network structure, EfficientNet deep network structure, and the like.

In step S12, a second image of the robot at the second time is obtained, and first expected position information of the robot at the second time is obtained based on the second image.

In an embodiment of the present invention, after obtaining the first position information of the robot at the first time, it is possible to continuously obtain the first expected position information of the robot at the second time. Here, a second image of the robot at the second time may be obtained using the camera, and the first expected position information of the robot at the second time may be obtained based on the second image. Referring specifically to FIG. 3, FIG. 3 is a flowchart illustrating an embodiment of step S12 of FIG. 1 according to an embodiment of the present invention, and specifically includes the following steps.

In step S121, the second reference object in the simulation sand table is obtained.

In step S122, a pixel position of the second reference object in a second image and a second pixel position of the robot are determined.

In step S123, a simulation position in the simulation sand table of the second reference object at the second time is determined.

In step S124, a second projection relationship is calculated and obtained using the pixel position of the second reference object and the simulated position of the second reference object in the simulation sand table.

In step S125, the simulation position of the robot in the simulation sand table is calculated and obtained based on the second projection relationship and the second pixel position of the robot.

In step S126, first expected position information of the robot at the second time is determined by using the simulation position of the robot in the simulation sand table.

Here, steps S121 to S126 are the same as the specific processes of steps S111 to S116 shown in FIG. 2 , and will not be described any further.

The robot positioning method provided in the embodiment of the present invention, in one aspect, through the method of a simulation sand table using a visual system, first obtains a projection relationship, and first position information of the robot at the first time and the second time obtaining first predicted location information in ; In another aspect, the dynamic system is used to obtain second expected position information of the robot at the second time through the initial direction of the robot, the movement distance within the second time from the first time, and the initial position information; In addition, the result of the visual system and the result of the dynamics system are fused through Kalman filtering to determine the second position information of the robot, that is, the final position information. Positioning is performed using the method of multi-terminal fusion, improving the accuracy of global positioning, reducing hardware cost, and facilitating the deployment of the system. In addition, when the first predicted position information is obtained through the visual system, there is no need to establish a simulation sand table and calibrate an additional reference object, thereby shortening the calibration time.

In step S13, the movement parameters of the robot in the process from the first time to the second time are collected.

In one embodiment of the present invention, the movement parameters of the robot are obtained in the process from the first time to the second time, and specifically referring to FIG. 4 , FIG. 4 is a diagram of FIG. 1 according to an embodiment of the present invention. It is a flowchart illustrating an embodiment of step S13, and specifically includes the following steps.

In step S131, a history motion parameter of the robot is acquired at the first time.

To understand, when the robot moves, the encoder inside the robot records the walking distance of the robot, and the historical walking distance of the robot is reflected in the steering gear. Accordingly, in one embodiment of the present invention, at a first time, the robot positioning device may record the robot's historical walking information, ie, the historical motion parameters, through the robot's encoder and steering gear. Here, while capturing the first image of the robot, historical motion parameters of the robot are acquired.

In step S132, a current motion parameter of the robot is acquired at the second time.

In one embodiment of the present invention, when the second time is reached, the encoder and steering gear information of the robot are read again to obtain the robot's current motion parameters.

Here, the historical motion parameter includes the historical motion distance of the robot, for example, may be the number of historical rotations of the wheels of the robot, and the current motion parameter may be the number of rotations of the wheels when the robot is at the second time.

In step S133, the motion parameters of the robot in the process from the first time to the second time are calculated and obtained using the historical motion parameters and the current motion parameters.

For example, assuming that the historical motion parameter of the robot at the first time is 100 meters and the current movement parameter detected at the second time is 101 meters, it is assumed that the robot moved a total of 1 meter from the first time to the second time. thing can be saved Alternatively, if it is assumed that the historical motion parameter of the robot at the first time is 200 rotations of the wheel, and the current motion parameter at the second time is 250 rotations of the wheel, the historical motion parameter from the first time to the second time It can be seen that the robot has rotated 50 times, that is, it can be obtained by calculating the total distance moved by the robot from the first time to the second time through the diameter of the wheel.

In an embodiment of the present invention, the robot has already obtained the direction at the first time, and at the same time knows the movement distance of the robot from the first time to the second time, and also obtains the first position information of the robot at the first time. If known, the second expected position information of the robot at the second time can be obtained. Specifically, reference is continued to FIG. 1 .

The robot positioning method provided in the embodiment of the present invention, in one aspect, through the method of a simulation sand table using a visual system, first obtains the projection relationship, and first position information of the robot at the first time and the second time obtaining first predicted location information in ; In another aspect, the dynamic system is further used to obtain second expected position information of the robot at the second time through the initial direction of the robot, the movement distance within the second time from the first time, and the initial position information; In addition, the result of the visual system and the result of the dynamics system are fused through Kalman filtering to determine the second position information of the robot, that is, the final position information. Using the positioning method of multi-terminal fusion to perform global positioning, not only improve accuracy, but also reduce hardware cost and facilitate system deployment. When the first predicted position information is obtained through the visual system, if the simulation sand table is established, there is no need to additionally calibrate the reference object, thereby shortening the calibration time.

In step S14, second expected position information of the robot at the second time is calculated and obtained based on the first position information and the motion parameter.

In an embodiment of the present invention, through the above steps, the direction of the robot at the first time is already known, and at the same time, the movement distance of the robot from the first time to the second time is also known, and the second time of the robot at the first time is also known. If the first position information is already known, it can be obtained by calculating the second expected position information of the robot at the second time through the dynamics system by combining it with the movement distance and direction based on the first position information.

In one embodiment, in order to detect the position information of the robot in real time, the interval from the first time to the second time is very short, for example, the interval time may be 1s. In another embodiment, when the direction change occurs in the process from the first time to the second time, the robot continuously takes pictures from the first time to the second time, Detect the direction of the robot based on the image. Here, when the robot is photographed, the movement distance of the robot can be detected at the same time, and when the direction of the robot is changed due to this, the movement distance in each direction can be recorded in real time.

It should be understood that, in order to accurately acquire the position of the robot, the interval time from the first time to the second time can be set as short as possible, so that the direction of the robot does not change in the process from the first time to the second time. can see.

In one embodiment, at least one camera may be installed in a place where the robot is located, the camera is connected to the robot positioning device, and a timer is set inside the robot positioning device or the camera, from the first time to the second time Set the interval time of the timer to the set time of the timer. When the set time is reached, the camera acquires the image of the robot and the movement parameters of the robot, and sends the image to the robot positioning device, which uses the vision system to use the vision system to obtain a second time through the method shown in FIG. Obtain the first expected position information of the robot at It is obtained by calculating the location information.

Here, the method of obtaining the first expected position information of the robot at the second time by using the second image is the same as the method of obtaining the first position information of the robot at the first time by using the first image, , the simulation sand table and the second image all contain a second reference; first obtain the pixel position of the second reference object and the second pixel position of the robot in the second image, and obtain the simulation position of the second reference object in the simulation sand table; calculating and obtaining a second projection relationship based on the pixel position of the second reference and the simulated position of the second reference; Then, through the second projection relationship and the second pixel position of the robot, the simulation position of the robot in the simulation sand table is calculated and obtained, in this way, using the simulation position of the robot in the simulation sand table, the second Determine the first expected position information of the robot at the time. For details, refer to FIG. 3, which will not be repeatedly described.

In step S15, the second position information of the robot at the second time is obtained through the first expected position information and the second expected position information.

In an embodiment, by performing a weighted average on the first predicted location information and the second predicted location information using the Kalman filtering method, the second location information of the robot at the second time may be obtained.

In another embodiment, the second location information may be obtained by fusing the first predicted location information and the second predicted location information through an average value statistical method or a Gaussian Mixture Model (GMM).

In the positioning method according to the embodiment of the present invention, by performing a weighted average on the first expected position information and the second expected position information using the Kalman filtering method, the second position information of the robot at the second time may be obtained. . Here, since the first expected position information obtained by the visual positioning system is already known, and the second expected position information obtained by the dynamics system is also known, first, state prediction with respect to the first expected position information and the second expected position information to obtain a covariance prediction result, and then perform an update on the covariance prediction result to obtain an updated covariance, thereby obtaining second position information through the updated covariance.

The present invention can obtain an estimate of the position of the extended Kalman filter at different time nodes through the evolution of discrete time.

As can be seen from this, the embodiment of the present invention obtains a robot positioning result with high accuracy by performing positioning result fusion through a combination of a visual positioning system and a dynamic positioning system. Referring to FIG. 5 , it is a diagram illustrating the principle of a robot positioning method according to an embodiment of the present invention, and the positioning process of the visual positioning system mainly includes the following processes. (1) reading the image with the camera; (2) performing vehicle detection with the detector YOLO-tiny, that is, performing detection of a vehicle, that is, a robot in the image, to obtain area coordinates in the image where the robot is located; (3) perform angle calculation with Mobilenet; That is, the image of the area where the robot is located is extracted and transmitted to the posture estimation module to implement the posture prediction; (4) Map postures and coordinates in the image to real-world coordinates using planar visual positioning, that is, calibration parameters. In another aspect, the positioning process of the dynamic positioning system mainly includes the following process. (1) obtain vehicle encoder and steering gear information; That is, based on the information, the historical motion parameter and the current motion parameter of the robot are obtained, that is, the robot's walking information is obtained; (2) predict the location through the vehicle dynamics model; That is, the current position information of the robot is predicted based on the historical motion parameter and the current motion parameter through the model. Next, accurate positioning of the robot is realized by fusing the results obtained through the video positioning system and the results obtained through the dynamics system through extended Kalman filtering.

The positioning method of the robot provided in the embodiment of the present invention, in one aspect, through a method of building a simulation sand table using a visual system, by first obtaining a projection relationship, obtaining location information and first expected location information at a second time; In another aspect, by using the dynamics system, obtaining second expected position information of the robot at the second time through the initial direction of the robot, the movement distance within the second time from the first time, and the initial position information; The result of the visual system and the result of the dynamics system are fused through the Kalman filter to determine the second position information of the robot, that is, the final position information. The positioning scheme uses the scheme of multi-terminal fusion to not only improve the accuracy of global positioning, but also reduce hardware costs and facilitate system deployment. When the first predicted position information is obtained through the visual system, if the simulation sand table is established, the calibration time is shortened because there is no need to calibrate the reference object additionally.

Referring to Figure 6, Figure 6 is a structural exemplary diagram of an embodiment of the robot positioning device of the present invention, the first position acquisition unit 41, the parameter acquisition unit 42, the second position acquisition unit 43 and the bridge government 44 .

Here, the first position acquiring unit 41 is configured to acquire the first position information and the first position of the robot at the first time, and the acquiring unit 41 is also configured to obtain the second image of the robot at the second time. is configured to obtain first expected position information of the robot at a second time based on the second image.

In some embodiments, when performing robot positioning, the positioning system may be directly used to obtain the first position information of the robot at the first time and the first expected position information of the robot at the second time, and positioning The system may be a GPS positioning system.

In some embodiments, it is also possible to acquire a first image of the robot at a first time through an imaging device such as a camera. Here, the robot is a movable mechanical equipment or robot positioning device, for example, a forklift, a mechanical trolley, or the like.

In some embodiments, a timer may be integrated in the camera, and when the set time of the timer reaches the first time, the camera takes a picture of the robot to obtain a first image. Here, the position of the camera may or may not be fixed; The shooting angle of the camera may or may not be fixed; Here, the camera may be installed at a specified position and not rotated, that is, the shooting range may be fixed, and the camera may also be installed and rotated at the specified position, that is, the reflection range may change; Of course, the camera can also be installed on a mobile device. The present invention does not limit the position and shooting range of the camera as long as the camera can take a picture of the robot.

In some embodiments, the first position acquisition unit 41 is further configured to acquire a first image of the robot at the first time point; and obtain the first position information of the robot at the first time based on the first image.

In some embodiments, the first image acquired by the camera may be uploaded to the robot positioning device, for example, the robot positioning device and the camera may be communicatively connected, and after the camera acquires the first image, the communication connection is established to send the first image to the robot positioning device via As long as it can be obtained, it is not limited thereto.

In some embodiments, the first position obtaining unit 41 is further configured to obtain a first reference in the simulation sand table, and determine the first position information of the robot through a manner of building the simulation sand table, and , it should be understood that the constructed simulation sand table is a simulation blueprint of the space where the robot is located; Determine a pixel position of a first reference in the first image and a first pixel position of the robot.

In some embodiments, by performing the identification on the first image using the first deep learning network, the pixel position of the first reference in the first image and the first pixel position of the robot may be determined.

In some embodiments, the first deep learning network may be a model integrating a deep learning network with a positioning function, by inputting a first image to the model, and the model performing identification on the first image, Obtain a pixel position of the first reference in the first image and a first pixel position of the robot in the first image.

Here, the first deep learning network is used to perform detection on the first image to determine the pixel position of the first reference in the first image and the first pixel position of the robot in the first image, and the first possible deep The learning network includes RCNN deep network structure, SSD deep network structure, Yolo deep network structure, RetinaNet network structure and the like.

Considering the cost and accuracy of location coordinate detection, the location coordinate detection can be performed through the Yolo deep network structure; Here, detection may be performed using the Yolo-tiny deep network structure in the Yolo deep network structure.

In the Yolo deep network architecture, the algorithm's mindset divides the whole image into some grids, predicting some possible bounding frames in the cell grid whose center is on the objects in the grid, and provides confidence, so that these algorithms detect objects in just one step. Since the result of the frame can be obtained, the speed of the algorithm is faster than the two-step Faster-RCNN series algorithm. Since the shape and color of the object to be detected in the application scenario are relatively fixed, the detection according to the small network structure for such a fast algorithm has high accuracy, and at the same time, because the occupied space of the computational resource is small, the mobile terminal calculation speed is relatively slow The required real-time detection effect can be achieved in CPU, for example Raspberry Pi, so the required cost is lower.

In some embodiments, it is necessary to determine the simulated position of the first reference through the pixel position of the first reference, and the first projection relationship using the pixel position of the first reference and the simulated position of the first reference in the simulation sand table is obtained by calculating

In some embodiments, the first location obtaining unit 41 is also configured to obtain the second reference in the simulation sand table. Determine a pixel position of the second reference object in the second image and a second pixel position of the robot. A simulation position of the second reference object in the simulation sand table at a second time is determined. A second projection relationship is calculated and obtained using the pixel position of the second reference object and the simulated position of the second reference object in the simulation sand table. The simulation position of the robot in the simulation sand table is calculated and obtained based on the second projection relationship and the second pixel position of the robot. First predicted position information of the robot at a second time is determined by using the simulation position of the robot in the simulation sand table.

The embodiment of the present invention builds a simulation sand table, performs the first projection relationship calculation according to using an object existing in the simulation sand table as a reference, and there is no need to set additional markers in the sand table, so the operation is simple; Identify the pixel position of the robot in the first image through deep learning, combine with the first projection relationship to determine the simulation position of the robot in the simulation sand table, and determine the first position information of the robot at the first time do. The positioning process of these robots simplifies the operation, reduces costs, and greatly improves the user experience.

In some embodiments, the first position acquiring unit 41 is further configured to detect the direction of the robot, that is, to detect the angle of the robot, when acquiring the first position information of the robot at the first time. Here, the direction of the robot may be detected through the angular posture prediction model.

Here, first performing identification on the first image by using the first deep learning network, and obtaining the position of the robot in the first image; Next, the area image where the robot is located is extracted, the extracted area image is input to the angle prediction model, the angle of the robot is detected through the angle prediction model, and the direction of the robot is obtained, where the direction of the robot is obtained After that, the movement direction of the robot within the second time from the first time can be known.

In some embodiments, a second deep learning network may be integrated into the angle prediction model, and the second deep learning network is used to identify the area image where the robot is located to determine the direction of the robot do. Here, the second deep learning network may be, for example, a convolutional network structure used for numerical regression in related technologies such as ResNet deep network structure, MobileNet deep network structure, GhostNet deep network structure, EfficientNet deep network structure, and the like.

In some embodiments, the parameter obtaining unit 42 is configured to collect the movement parameters of the robot in the process from the first time to the second time. Here, first, the history motion parameters of the robot are acquired at the first time.

When the robot moves, the encoder inside the robot records the walking distance of the robot, and the historical walking distance of the robot is reflected in the steering gear. Therefore, at the first time, the robot positioning device records the historical walking information of the robot, that is, the historical motion parameters, through the encoder and the steering gear of the robot. Here, while capturing the first image of the robot, historical motion parameters of the robot are acquired.

Next, it continues to acquire the current motion parameter of the robot at the second time point. Here, when the second time is reached, the robot positioning device reads the encoder and steering gear information of the robot again to obtain the robot's current movement parameters.

Here, the historical motion parameter includes the historical motion distance of the robot, for example, may be the number of historical rotations of the wheels of the robot, and the current motion parameter may be the number of rotations of the wheels when the robot is at the second time. The motion parameters of the robot in the process from the first time to the second time are calculated and obtained through the historical motion parameters and the current motion parameters. Here, assuming that the historical motion parameter of the robot at the first time is 100 meters and the current motion parameter detected at the second time is 101 meters, it is calculated that the robot moved a total of 1 meter from the first time to the second time. can Alternatively, in another embodiment, assuming that the historical motion parameter of the robot at the first time is 200 revolutions of the wheel and the current motion parameter at the second time is 250 revolutions of the wheel, It can be seen that the robot rotated 50 times from the first time to the second time, and it can be obtained by calculating the total distance the robot moved from the first time to the second time through the diameter of the wheel, etc.

The second position acquiring unit 43 is configured to calculate and obtain second expected position information of the robot at the second time based on the first position information and the motion parameter. Here, when the robot already knows the direction at the first time, also knows the movement distance of the robot from the first time to the second time, and already knows the first position information of the robot at the first time, the first Based on the position information, combined with the movement distance and direction, it is possible to calculate and obtain second expected position information of the robot at the second time through the dynamics system.

In some embodiments, in order to detect the position information of the robot in real time, the interval from the first time to the second time is very short, for example, the interval time may be 1s. In another embodiment, when the direction change occurs in the process from the first time to the second time, the robot continuously takes pictures from the first time to the second time, Detect the direction of the robot based on the image. Here, when the robot is photographed, the movement distance of the robot can be detected at the same time, and when the direction of the robot is changed due to this, the movement distance in each direction can be recorded in real time.

It should be understood that, in order to accurately acquire the position of the robot, the interval time from the first time to the second time can be set as short as possible, so in this way, the direction of the robot in the process from the first time to the second time It can be seen that this does not change.

In some embodiments, at least one camera may be installed at the location where the robot is located, the camera is connected to the robot positioning device, and a timer is set inside the robot positioning device or the camera, from a first time to a second time Set the interval time of the timer to the set time of the timer. When the set time is reached, the camera acquires the image of the robot and the movement parameters of the robot, and sends the image to the robot positioning device, which uses the vision system to use the vision system to obtain a second time through the method shown in FIG. Obtain the first expected position information of the robot at It is obtained by calculating the location information.

The calibration unit 44 is configured to obtain the second position information of the robot at the second time through the first expected position information and the second expected position information. In some embodiments of the present invention, by performing a weighted average on the first expected position information and the second expected position information using a Kalman filtering method, the second position information of the robot at the second time can be obtained

In some other exemplary embodiments, the second location information may be obtained by fusing the first predicted location information and the second predicted location information through an average value statistical method or a Gaussian mixed model (GMM).

The robot positioning method provided in the embodiment of the present invention, in one aspect, through a method of building a simulation sand table using a visual system, by first obtaining a projection relationship, the first position information of the robot at the first time and obtain first expected location information at a second time; In another aspect, by using the dynamics system, obtaining second expected position information of the robot at the second time through the initial direction of the robot, the movement distance within the second time from the first time, and the initial position information; The result of the visual system and the result of the dynamics system are fused through the Kalman filter to determine the second position information of the robot, that is, the final position information. The positioning scheme uses the scheme of multi-terminal fusion to not only improve the accuracy of global positioning, but also reduce hardware costs and facilitate system deployment. When the first predicted position information is obtained through the visual system, if the simulation sand table is established, the calibration time is shortened because there is no need to calibrate the reference object additionally.

Referring to FIG. 7, FIG. 7 is a structural diagram of an embodiment of a robot positioning device according to an embodiment of the present invention. It includes a memory 52 and a processor 51 connected to each other.

The memory 52 is configured to store program instructions for implementing any of the above robot positioning methods.

The processor 51 is configured to execute program instructions stored in the memory 52 .

Here, the processor 51 may also be referred to as a CPU. The processor 51 may be an integrated circuit chip having signal processing capability. The processor 51 may also be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device; It can be a discrete gate or transistor logic device, a discrete hardware assembly. A general-purpose processor may be a microprocessor or any general processor or the like.

The memory 52 may be a memory stick, a TF card, or the like, and may store all information of the robot positioning device, and may store input raw data, computer program, intermediate driving results, and final driving results all in the memory. Memory 52 stores and retrieves information according to a location designated by the controller. Only when there is memory, the robot positioning device can have a memory function and ensure normal operation. The memory of the robot positioning device can be divided into memory main memory (internal storage) and auxiliary memory (external storage) according to the purpose, and there is also a method of classifying it into external memory and internal memory. External storage is generally a magnetic medium or optical disk that can store information for a long period of time. Internal storage refers to the storage component of the motherboard, is used to store the currently running data and programs, only temporarily stores programs and data, and when the power is turned off or turned off, the data is lost.

An embodiment of the present invention provides a computer program comprising computer readable code, and when the computer readable code is operated on a robot positioning device and executed by a process of the robot positioning device, implements the robot positioning method .

In the various embodiments provided by the present invention, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the device embodiment described above is merely exemplary, for example, the partition of a module or unit is only a logical function partition, and there may be other partitioning methods in actual application, for example, a plurality of units Alternatively, components may be combined or integrated into another system, or some features may be omitted or not implemented. Further, any coupling or direct coupling or communication connection between each other shown or discussed may be implemented through some interface, and the indirect coupling or communication connection through a device or unit may be electrical, mechanical, or other form.

A unit described as a separation member may or may not be physically separated, and a member indicated as a unit may or may not be a physical unit, and may be located in one location or distributed in a plurality of network units. According to actual needs, some or all of the units may be selected to implement the purpose of the scheme of the present embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may be an independent physical entity, and two or two or more units may be integrated into one unit. there is. The integrated unit may be implemented using a form of hardware or may be implemented using a form of a software functional unit.

When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, that is, a part contributing to the prior art or a part of the technical solution may be implemented in the form of a software product, the computer software product being stored in one storage medium, one It includes a plurality of instructions used to cause a computer device (which may be a personal computer, server or network device, etc.) or processor of the present invention to execute all or some steps of the method of each embodiment of the present invention.

Referring to FIG. 8 , it is an exemplary structural diagram of a computer-readable storage medium according to an embodiment of the present invention. The storage medium of the present invention stores a program file 61 that can implement all the robot positioning methods, wherein the program file 61 can be stored in the storage medium in the form of a software product, and a computer device ( It may be a personal computer, server, or network device, etc.) or includes some instructions for causing a processor to execute some steps of the methods described in the embodiments of the present invention. The above-mentioned storage device is a U disk, a removable hard disk, ROM (Read-Only Memory, ROM), RAM (Random Access Memory, RAM), a variety of media that can store a program code such as a magnetic disk or an optical disk, or a computer, server , including terminal devices such as mobile phones and tablets.

The above is only a preferred embodiment of the present invention, it does not limit the scope of the present invention, it is an equivalent structure or equivalent process modification using the contents of the specification and drawings of the present invention, or directly or indirectly operating in other related technical fields and are also included within the scope of patent protection of the present invention.

Industrial Applicability

In an embodiment of the present invention, first position information of the robot at a first time is obtained, a second image of the robot at a second time is obtained, and based on the second image at a second time acquiring first expected position information of the robot; collect motion parameters of the robot in the process from the first time to the second time; calculating and obtaining second expected position information of the robot at the second time based on the first position information and the motion parameter; The second position information of the robot at the second time is obtained through the first expected position information and the second expected position information. By fusing the robot positioning results obtained through two different methods, accurate position information is obtained, and positioning accuracy is effectively improved.

Claims (23)

A robot positioning method comprising:
acquiring first position information of the robot at a first time;
obtaining a second image of the robot at a second time, and obtaining first expected position information of the robot at the second time based on the second image;
collecting motion parameters of the robot in the process from the first time to the second time;
calculating and obtaining second expected position information of the robot at the second time based on the first position information and the motion parameter; and
and obtaining second position information of the robot at the second time through the first expected position information and the second expected position information. According to claim 1,
Acquiring the first position information of the robot at the first time comprises:
acquiring a first image of the robot at the first time; and
and acquiring the first position information of the robot at the first time based on the first image. 3. The method of claim 2,
The first image includes a first reference, and the step of obtaining the first position information of the robot at the first time based on the first image comprises:
obtaining the first reference in a simulation sand table;
determining a pixel position of the first reference in the first image and a first pixel position of the robot;
determining a simulation position of the first reference object in the simulation sand table at a first time;
calculating and obtaining a first projection relationship using the pixel position of the first reference and the simulated position of the first reference in the simulation sand table;
calculating and obtaining a simulated position of the robot in the simulation sand table based on the first projection relationship and a first pixel position of the robot; and
and determining first position information of the robot at the first time by using the simulated position of the robot in the simulation sand table. According to claim 1,
The second image includes a second reference, and the step of obtaining the first expected position information of the robot at the second time based on the second image comprises:
obtaining the second reference in a simulation sand table;
determining a pixel position of the second reference object and a second pixel position of the robot in the second image;
determining a simulation position of the second reference object in the simulation sand table at a second time;
calculating and obtaining a second projection relationship using the pixel position of the second reference object and the simulated position of the second reference object in the simulation sand table;
calculating and obtaining a simulation position of the robot in the simulation sand table based on the second projection relationship and a second pixel position of the robot; and
and determining the first expected position information of the robot at the second time by using the simulated position of the robot in the simulation sand table. 5. The method of claim 4,
Determining the pixel position of the second reference object and the second pixel position of the robot in the second image comprises:
performing identification on the second image using a first deep learning network to determine a pixel position of the second reference object and a second pixel position of the robot in the second image;
The first deep learning network includes one or any combination of a Region-CNN (RCNN) deep network structure, a Single Shot MultiBox Detector (SSD) deep network structure, a You Only Look Once (Yolo) deep network structure, and a RetinaNet network structure. Robot positioning method, characterized in that. 3. The method of claim 2,
After acquiring the first image of the robot at the first time, the method comprises:
The robot positioning method, characterized in that it further comprises the step of obtaining the direction of the robot at the first time based on the first image. 7. The method of claim 6,
Acquiring the direction of the robot at the first time based on the first image comprises:
and determining the direction of the robot by performing identification on the image of the area where the robot is located using a second deep learning network,
The second deep learning network is a robot positioning method, characterized in that it comprises one or any combination of ResNet deep network structure, MobileNet deep network structure, GhostNet deep network structure, EfficientNet deep network. 7. The method of claim 6,
The positioning method is
Further comprising the step of obtaining the historical motion parameter of the robot at the first time,
Collecting the movement parameters of the robot in the process from the first time to the second time,
acquiring a current motion parameter of the robot at the second time;
and calculating and obtaining a motion parameter of the robot in the process from a first time to a second time through the historical motion parameter and the current motion parameter. 7. The method of claim 6,
Calculating and obtaining second expected position information of the robot at the second time based on the first position information and the motion parameter includes:
Through the movement parameter of the robot in the process from the first time to the second time, it is combined with the direction of the robot in the first time to obtain second expected position information of the robot at the second time Robot positioning method comprising the step of. According to claim 1,
The step of obtaining the second position information of the robot at the second time through the first expected position information and the second expected position information comprises:
Obtaining second position information of the robot at the second time by performing a weighted average on the first expected position information and the second expected position information using a Kalman filtering method Robot positioning method, characterized in that. A robot positioning device comprising:
Obtaining first position information of the robot at a first time, obtaining a second image of the robot at a second time, and obtaining a first expected position of the robot at the second time based on the second image a first location obtaining unit configured to obtain information;
a parameter acquisition unit configured to collect movement parameters of the robot in a process from a first time to a second time;
a second position acquiring unit configured to calculate and obtain second expected position information of the robot at the second time based on the first position information and the motion parameter;
and a calibration unit configured to obtain second position information of the robot at the second time through the first expected position information and the second expected position information. 12. The method of claim 11,
The first position acquiring unit is specifically configured to acquire a first image of the robot at the first time; and obtain the first position information of the robot at the first time based on the first image. 13. The method of claim 12,
The first image includes a first reference,
The first location obtaining unit is further specifically configured to: obtain the first reference in the simulation sand table; determine a pixel position of the first reference in the first image and a first pixel position of the robot; determine a simulation position of the first reference object in the simulation sand table at a first time; calculating and obtaining a first projection relationship using the pixel position of the first reference and the simulated position of the first reference in the simulation sand table; calculating and obtaining a simulation position of the robot in the simulation sand table based on the first projection relationship and the first pixel position of the robot; and determine the first position information of the robot at the first time by using the simulated position of the robot in the simulation sand table. 12. The method of claim 11,
the second image includes a second reference,
The first location obtaining unit is also specifically configured to obtain the second reference in the simulation sand table; determine a pixel position of the second reference object and a second pixel position of the robot in the second image; determine a simulation position of the second reference object in the simulation sand table at a second time; calculating and obtaining a second projection relationship using the pixel position of the second reference object and the simulated position of the second reference object in the simulation sand table; calculating and obtaining a simulation position of the robot in the simulation sand table based on the second projection relationship and a second pixel position of the robot; and determine the first expected position information of the robot at the second time by using the simulated position of the robot in the simulation sand table. 15. The method of claim 14,
The first position obtaining unit is further specifically configured to perform identification on the second image using a first deep learning network, so that the pixel position of the second reference object in the second image and the second pixel of the robot one of the first deep learning network RCNN (Region-CNN) deep network structure, SSD (Single Shot MultiBox Detector) deep network structure, Yolo (You Only Look Once) deep network structure, RetinaNet network structure. Or robot positioning device comprising any combination. 12. The method of claim 11,
The first position obtaining unit is further specifically configured to obtain a first image of the robot at the first time, and then obtain a direction of the robot at the first time based on the first image Robot positioning device, characterized in that. 17. The method of claim 16,
The first position obtaining unit is further specifically configured to determine the direction of the robot by performing identification on the image of the area where the robot is located using a second deep learning network, and the second deep learning network is A robot positioning device comprising one or any combination of ResNet deep network architecture, MobileNet deep network architecture, GhostNet deep network architecture, and EfficientNet deep network. 17. The method of claim 16,
The parameter obtaining unit is specifically configured to obtain a historical motion parameter of the robot at the first time, and the parameter obtaining unit is further configured to obtain a current motion parameter of the robot at the second time; and acquiring the motion parameter of the robot in the process from the first time to the second time through the historical motion parameter and the current motion parameter. 17. The method of claim 16,
The second position obtaining unit may be specifically configured to combine with the direction of the robot within the first time through the movement parameter of the robot in the process from the first time to the second time, and the second time at the second time. Robot positioning device, characterized in that configured to obtain the second expected position information of the robot. 12. The method of claim 11,
The calibration unit is specifically configured to obtain the second position information of the robot at the second time through the first expected position information and the second expected position information. A robot positioning device comprising: a memory and a processor, the memory storing program instructions, the processor calling the program instructions from the memory; A robot positioning device, characterized in that it runs. A computer-readable storage medium storing a program file, the program file being executable to implement the robot positioning method according to any one of claims 1 to 10. A computer program comprising computer readable code, wherein the computer readable code is operated on a robot positioning device and executed by a processor of the robot positioning device. A computer program comprising implementing a method.

Images