WO2023097577A1

WO2023097577A1 - Expandable relative positioning device based on uwb and camera, and method

Info

Publication number: WO2023097577A1
Application number: PCT/CN2021/134852
Authority: WO
Inventors: 曹燕军; 宣驰策
Original assignee: 浙江大学湖州研究院
Priority date: 2021-12-01
Filing date: 2021-12-01
Publication date: 2023-06-08

Abstract

An expandable relative positioning device (A) based on an UWB and a camera. The device (A) comprises: an UWB module (2), a camera module (3), a light-emitting module (4) and a calculation unit (1) which are mounted on a body, wherein the UWB module (2) is used for measuring absolute distance measurement information between the UWB module (2) and an UWB module (2) of at least another adjacent device (B); the camera module (3) is used for measuring absolute angle measurement information of the at least another device (B); and the calculation unit (1) collects in real time the absolute distance measurement information obtained by the UWB module (2) and the absolute angle measurement information obtained by the camera module (3), and is configured to obtain distance and orientation information of the at least another adjacent device (B) on the basis of the absolute distance measurement information and the absolute angle measurement information. Further disclosed is a method for performing relative positioning using a relative positioning device (A).

Description

An expandable device and method for relative positioning based on UWB and camera

technical field

The invention relates to a device for relative positioning, more specifically, an expandable device for relative positioning based on UWB and a camera, and a method for relative positioning of the device.

Background technique

Relative positioning is very important in multi-robot systems and multi-agent systems. For example, multiple unmanned vehicles perceive the position of adjacent unmanned vehicles in their own coordinate system. It can also be used for multiple robots, handheld devices, and drones. , VR/AR equipment and other systems. Obtaining relative position information is very helpful for navigation, obstacle avoidance, and planning, such as autonomous following, cluster cooperation, and virtual reality.

The current schemes for obtaining relative orientation can be divided into two types, from global to relative methods and direct relative observation methods. The global-to-local method requires that the relevant equipment first have global coordinates, that is, the pose in a unified coordinate system, and then obtain the relative orientation by comparing the global coordinates, such as GPS, multi-base station-based methods, and multi-robot collaborative SLAM. Solutions such as global GPS, WIFI signal fingerprints, and multi-base stations (such as UWB, RFID, Bluetooth) usually require more preparation work, pre-established base stations or data collection, and the scope of use is strictly limited to predetermined areas. The control center calculates through the global base station Find the position of the individual and then get the relative orientation between the individuals. SLAM used in the field of robotics can also achieve relative orientation observation from global to relative. The SLAM real-time positioning and mapping solution realizes accurate self-state estimation and environmental maps by simultaneously estimating the features in the environment and its own motion information. Multiple robots can obtain a unified map by sharing maps and closed-loop detection between robots. And the position of each robot, from which the relative orientation is calculated. Such schemes all rely on a control center, pre-built facilities, or individuals require a large amount of calculations, without direct measurement of relative orientation, and are not suitable for rapid deployment of large-scale robotic systems.

The relative orientation information includes distance information and relative angle information. Direct measurement of distance usually relies on reflected waves of lasers, reflected waves of ultrasonic waves, and transmission and reception of radio waves. Laser ranging needs to emit a very directional laser and receive reflected light to measure the distance. The ultrasonic ranging method is similar. The high requirement for directivity of this method makes it difficult to apply in the field of mobile robotics. Radio waves measure distances by transmitting and receiving electromagnetic waves. Usually, back-and-forth transmission and TOF are used to measure distances. UWB transmits and receives ultra-short broadband pulse waves, and uses TOF to measure distances between nodes. However, it is still not possible. Direct observation of the relative angle.

The latest Bluetooth 5.1 technology can realize the observation of the angle of arrival of the signal by customizing a special array antenna, but the accuracy that can be achieved is limited, and the observation of the distance is lacking.

In multi-robot applications or related positioning applications, there is no method that can directly observe the relative orientation between robots. The SLAM method requires high computing resources and is expensive, and the solution based on infrastructure such as multiple base stations requires additional equipment deployment work, which is not suitable for the application of mobile robots. When used for relative pose estimation, the above two methods calculate the relative position after obtaining the global coordinates, resulting in useless computing power consumption and large time delay, and the system cannot be expanded.

UWB single-point ranging can obtain centimeter-level accuracy, but relative orientation information cannot be easily obtained. The new technology of Bluetooth 5.1 can obtain the angle of arrival of the signal, but the accuracy is limited and a special antenna is required. Laser ranging can obtain high-precision distance measurement, but requires a highly directional beam, and is not suitable for applications on mobile robots.

Contents of the invention

The technical problem to be solved by the present invention is to provide an expandable relative positioning technology, which can enable relatively fast and convenient relative positioning of equipment with high positioning accuracy.

In order to achieve the above object, the present invention provides an expandable relative positioning device based on UWB and camera, which is characterized in that the device includes: a UWB module installed on the body, a CAMERA module, a light emitting module and a computing unit, wherein The UWB module is used to measure the absolute distance measurement information between the UWB module of at least another adjacent device; the CAMERA module is used to measure the absolute angle measurement information of the at least another device; and the calculation unit is real-time Collecting the absolute distance measurement information obtained by the UWB module and the absolute angle measurement information obtained by the CAMERA module, and configured to obtain the distance and the distance of the at least another adjacent device according to the absolute distance measurement information and the absolute distance measurement information orientation information.

Further, the light emitting module is an LED module.

Further, the computing unit is configured to control the light emitting module to emit coded light coded with ID information of the device in real time.

Further, the CAMERA module of the device is configured to capture the coded light of the light emitting module of the at least another adjacent device, identify the ID information of the at least another adjacent device through code comparison, and obtain the at least another adjacent device. The pixel coordinate position of one neighboring device on the pixel plane of the device, and the direction information of the at least another neighboring device in the camera coordinate system of the device is obtained according to the camera model.

Further, the direction information includes yaw angle and pitch angle.

Further, the light emitting module is installed within the FOV range of the CAMERA module, and the FOV of the CAMERA module is above 180°.

Further, the CAMERA module is equipped with two cameras with FOV greater than 180° on the front and rear sides of the body.

The present invention also provides an expandable method for relative positioning of multiple above-mentioned devices based on UWB and camera, which is characterized in that: the UWB module of one device measures the absolute distance measurement information between the UWB of at least another adjacent device , the CAMERA module of the device measures the absolute angle measurement information of the at least one other device, and the calculation unit of the device collects the absolute distance measurement information obtained by the UWB module and the absolute angle obtained by the CAMERA module in real time measuring information, and obtaining distance and orientation information of the at least another adjacent device according to the absolute distance measurement information and the absolute distance measurement information.

Further, the calculation unit controls the light emitting module in real time to emit encoded light encoded with the ID information of the device.

Further, the CAMERA module of the device captures the coded light of the light emitting module of the at least another neighboring device, and identifies the ID information of the at least another neighboring device through code comparison, and obtains the ID information of the at least another neighboring device. The pixel coordinate position of the device on the pixel plane of the device, and obtain the direction information of the at least another adjacent device in the camera coordinate system of the device according to the camera model.

Furthermore, when multiple modules appear around a device, since the luminescence code of each LED module is unique, the device can effectively locate different modules in the field of view from the sequence of pictures.

Beneficial effect

The invention uses the UWB module of a single device to measure the absolute distance measurement information and the absolute angle measurement information based on the camera, and realizes an expandable relative measurement method. By making a relative measurement module or integrating it in other systems, multi-robots can be realized System and multi-machine system co-location.

In social applications, the present invention can be used in various scenarios of indoor positioning and outdoor positioning, and each module can accurately determine the spatial position of adjacent equipment, so it can be used for electronic tags in factories, hospitals, nursing homes, prisons and other scenarios Positioning can be quickly deployed in sports fields, booths and other scenes where it is not suitable to install additional external facilities. It can also be integrated into VR/AR equipment, so that multiple users can obtain information corresponding to the relative orientation of reality in the virtual environment, which has great advantages. great economic value. At the same time, the invention can be fully used in multi-robot system positioning, the system does not depend on GPS, can provide reliable relative orientation measurement, and improve the robustness of the system.

Description of drawings

Fig. 1 has four module connection block diagrams among the present invention;

Fig. 2 is a schematic diagram of the interaction mode of two devices in the working process in the present invention;

Fig. 3 is a schematic diagram of the relative positions of two devices in Embodiment 1 of the present invention;

Fig. 4 is a schematic plan view of the camera of module A in Embodiment 1 of the present invention;

FIG. 5 is a schematic structural diagram of configuring two cameras with an FOV greater than 180° in Embodiment 2 of the present invention;

FIG. 6 is a schematic diagram of a positioning effect of an omnidirectional neighbor module that can realize a 360° spherical field of view in Embodiment 2 of the present invention;

7 is a schematic structural diagram of a device using a UWB module with a built-in onboard antenna in Embodiment 3 of the present invention;

Fig. 8 is a schematic diagram of the effect in Embodiment 4 of the present invention.

Detailed ways

The implementation of the present invention is described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

A kind of extensible relative positioning method based on UWB and camera of the present invention includes UWB (Ultra Wide Band, ultra-wideband) module 2, CAMERA (camera) module 3, light emitting (such as LED) module 4 and computing unit 1 equipment (as shown in Figure 1). Each device with the four modules can obtain the communication range of the UWB module 2 through the absolute distance measurement information of the UWB module 2 and the absolute angle measurement information based on the CAMERA module 3, or within the perception range of the CAMERA module 3 and the LED module 4 The distance and orientation information of the neighboring modules.

During the working process, the calculation unit 1 collects the data of the UWB module 2 and the CAMERA module 3 at all times, and controls its own LED module 4 to emit structured light encoded with its own ID information at all times.

As shown in Figure 2, it expresses the interaction mode of the two devices during the working process. The UWB module of the A device and the UWB module of the B device are continuously measured to obtain centimeter-level accuracy distance information. The distance information between the device A and the device B all know the distance information between them. At the same time, the CAMERA module between the devices will capture the light status of the LED module of the adjacent module, and the adjacent module can be identified through code comparison, and then according to the pixel position of the LED module information on the pixel plane and the camera model, the adjacent module is Orientation information in the own camera coordinate system. For example, the CAMERA module of device A can capture the coded light of the LED module of device B, and then through the processing and analysis of the picture, the position of device B can be found on the picture, and then the relative position of device B to the camera module of device A can be calculated through the camera model. direction. By combining the relative distance obtained by the UWB module, the relative orientation between devices can be obtained, that is, device A can accurately know the position of device B in its coordinate system, and similarly, device B can also know the position of device A in its coordinate system.

When there are multiple modules around a device, since the light-emitting code of each LED module is unique, the device can effectively locate different modules in the field of view from the picture sequence, thus realizing simultaneous identification of multiple adjacent devices function, and then fuse the distance information of the UWB module, the relative orientation of all adjacent devices can be efficiently obtained, and an expandable relative orientation measurement solution can be realized.

Embodiment one

As shown in FIG. 3 , there are two devices, A device 5 and B device 6 . Each device includes a body on which the above-mentioned UWB module 2, CAMERA module 3, light emitting module 4 and computing unit 1 are installed. The UWB module 2 is a UWB antenna protruding from the upper side of the body, the CAMERA module 3 is a camera installed on one side of the body, and the light emitting module 4 is an LED module installed within the FOV range of the CAMERA module 3 .

Next, the orientation of the B device 6 in the A coordinate system will be analyzed by taking the A device 5 as an object. The UWB ranging modules of the A device 5 and the B device 6 can measure the distance d from O_A to O_B. The camera plane of A device 5 is shown in Figure 4. B device 6 is within the viewing angle of A. The coordinates on the A camera plane are (u, v). The B device can be obtained through the pixel coordinates (u, v) and the camera model. 6 The yaw angle and pitch angle of the O_A-x-y coordinate system of the A device 5 not only obtain the direction of the B device 6, but also obtain the position of the B device in the A device coordinate system by combining the distance measurement of the UWB. Similarly, the B-device 6 can also obtain the orientation of the A-device 5 in its coordinate system through the pixel position of the A-device 5 in the camera plane.

Embodiment two

As shown in Figure 5, the CAMERA module 3 can realize a 360° spherical field of view by disposing two cameras with FOV (field of view) greater than or equal to 180° on the front and rear sides of the main body respectively, and realize the positioning of omnidirectional neighbor modules (such as Figure 6).

Embodiment Three

As shown in Figure 7, by adopting a UWB module with a built-in on-board antenna, a compact relative positioning device can be realized while reducing the cost.

Embodiment Four

As shown in Figure 8, it shows the usage scenario of relative positioning of multiple modules, which is suitable for positioning other modules in the environment with only one top-configured module in the scenario, such as the application of positioning helmets in the factory scenario. At the same time, each module is the same and can be used for relative positioning algorithms between multiple robots.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims

An expandable relative positioning device (A) based on UWB and camera, characterized in that the device includes: a UWB module (2), a CAMERA module (3), a light emitting module (4) and computing unit (1), where

The UWB module (2) is used to measure the absolute distance measurement information between the UWB module of at least another adjacent device (B);

said CAMERA module (3) for measuring absolute angle measurement information of said at least one other device (B); and

The calculation unit (1) collects the absolute distance measurement information obtained by the UWB module (2) and the absolute angle measurement information obtained by the CAMERA module (3) in real time, and is configured to The measurement information obtains distance and orientation information of said at least one other neighboring device (B).
The device (A) according to claim 1, characterized in that the light emitting module (4) is an LED module.
The device (A) according to claim 1 or 2, characterized in that the computing unit (1) is configured to control the light emitting module (4) in real time to emit coded light encoded with ID information of the device.
The device (A) according to claim 3, characterized in that: the CAMERA module (3) of the device (A) is configured to capture the light emitting module (4) of the at least another adjacent device (B) Coded light, identifying the ID information of the at least another adjacent device (B) through code comparison, and obtaining the pixel coordinate position of the at least another adjacent device (B) on the pixel plane of the device (A) , and obtain the direction information of the at least another neighboring device (B) in the camera coordinate system of the device (A) according to the camera model.
The device (A) according to claim 4, wherein the direction information includes yaw angle and pitch angle.
The device (A) according to claim 1 or 2, wherein the light-emitting module (4) is installed within the FOV range of the CAMERA module (3), and the FOV of the CAMERA module (3) is 180 ° above.
The device (A) according to claim 6, characterized in that the CAMERA module (3) is respectively equipped with two cameras with FOV greater than 180° on the front and rear sides of the body.
An expandable method for relative positioning of multiple devices based on UWB and camera, each of the devices is the device according to any one of claims 1-7, characterized in that:

Absolute distance measurement information between the UWB module (2) of one of the devices (A) and the UWB module (2) of at least one other adjacent device (B),

the CAMERA module (3) of said device (A) measures absolute angle measurement information of said at least one other device (B), and

The calculation unit (1) of the device (A) collects the absolute distance measurement information obtained by the UWB module (2) and the absolute angle measurement information obtained by the CAMERA module (3) in real time, and according to the absolute distance measurement information and absolute distance measurement information to obtain distance and orientation information of said at least one other neighboring device (B).
The method according to claim 8, characterized in that the computing unit (1) controls the light emitting module (4) in real time to emit encoded light encoded with the ID information of the device.
The method according to claim 9, characterized in that: the CAMERA module (3) of the device (A) captures the coded light of the light emitting module (4) of the at least another adjacent device (B), and compares the coded light Identifying the ID information of the at least another adjacent device (B), obtaining the pixel coordinate position of the at least another adjacent device (B) on the pixel plane of the device (A), and obtaining according to the camera model Obtaining the direction information of the at least another neighboring device (B) in the camera coordinate system of the device (A).