WO2022078512A1

WO2022078512A1 - Map establishment method and apparatus, and self-moving device and storage medium

Info

Publication number: WO2022078512A1
Application number: PCT/CN2021/124196
Authority: WO
Inventors: 高梓翔
Original assignee: 北京猎户星空科技有限公司
Priority date: 2020-10-16
Filing date: 2021-10-15
Publication date: 2022-04-21
Also published as: CN111932675A; CN111932675B

Abstract

Provided are a map establishment method and apparatus, and a self-moving device and a storage medium, which relate to the field of automatic navigation. The method comprises: acquiring an image collected by a self-moving device during the moving process, and a device pose of the self-moving device in a map coordinate system during image collection; decoding an encoded portion in a position identifier according to a coordinate system portion in the position identifier in the image, so as to obtain a target code; according to the target code, performing querying to obtain a standard coordinate position of the encoded portion in a standard coordinate system in which the position identifier is located; according to the standard coordinate position and a reference coordinate position, determining a coordinate conversion relationship between the standard coordinate system and a coordinate system of the self-moving device during image collection; and according to the coordinate conversion relationship and the device pose of the self-moving device in the map coordinate system, marking a predicted pose of the position identifier in the map coordinate system. Therefore, the effectiveness of an established map can be ensured, such that the position of a self-moving device can be effectively located, thereby improving the applicability thereof.

Description

Method, device, self-moving device and storage medium for building a map

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure claims the priority of the Chinese patent application number "202011106961.2", filed by Beijing Orion Star Technology Co., Ltd. on October 16, 2020, with the invention titled "Method, Apparatus, Self-Moving Device and Storage Medium for Establishing a Map".

technical field

The present disclosure relates to the technical field of automatic navigation, and in particular, to a method, an apparatus, a self-moving device and a storage medium for establishing a map.

Background technique

With the continuous development of artificial intelligence technology, artificial intelligence products, such as self-mobile devices, continue to be popularized. When controlling the self-mobile device to move, it is necessary to locate the self-mobile device first, that is, to identify the position of the self-mobile device in the space, and then the self-mobile device can be navigated.

In the related art, a map navigation method is used to determine the position of the self-mobile device in the indoor space. Specifically, a map is pre-established, and the self-mobile device is equipped with a laser radar. During the movement of the self-mobile device, the laser radar is used to scan the surrounding environment to obtain a laser point cloud image, and the point cloud image collected according to the laser radar is collected. The cloud map is created from the location of the mobile device. Furthermore, during navigation and positioning, the real-time position of the self-mobile device in the map can be determined by matching the point cloud image collected by the lidar in real time with the pre-established map, so as to realize the positioning of the self-mobile device.

SUMMARY OF THE INVENTION

The present disclosure proposes a method, device, self-moving device and storage medium for establishing a map, so as to realize the establishment of a map according to a location identifier, even if the display position in the space where the self-moving device is located changes, or the movement of personnel interferes with the scanning of the surrounding by the lidar environment, the established map will not fail, so that the position of the self-mobile device can be effectively located, and the applicability of the method can be improved. It is used to solve the problem in the related art by using the lidar to scan the surrounding environment during the movement of the self-mobile device. Obtaining a laser point cloud image, based on the point cloud image collected by lidar, and the method of building a map from the location of the mobile device when collecting the point cloud image has the technical problem that it cannot be applied to scenes where the indoor environment frequently changes.

The embodiment of the first aspect of the present disclosure provides a method for establishing a map, including:

Obtaining an image collected from a mobile device during the movement process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; wherein, the image displays a position identifier;

According to the coordinate system part in the position identification, decode the coding part in the position identification to obtain the target code;

According to the target code, query to obtain the standard coordinate position of the code part in the standard coordinate system where the position identifier is located;

According to the standard coordinate position and the reference coordinate position, determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is captured; wherein, the reference coordinate position is the code the coordinate position of the part in the candidate coordinate system determined by the coordinate system part;

According to the coordinate transformation relationship and the device pose of the self-moving device in the map coordinate system, the predicted pose of the location identifier is marked in the map coordinate system.

In a first possible implementation manner of the embodiment of the present disclosure, the images showing the same location identifier are at least two frames, and the self-mobile device has a corresponding device in the map coordinate system when collecting images of each frame pose;

Wherein, according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system, marking the predicted pose of the location identifier in the map coordinate system includes:

According to the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when each frame of the image is collected, and according to the device pose of the self-mobile device in the map coordinate system when each frame of image is collected , in the map coordinate system, determine the observation pose of the position marker corresponding to each frame of the image;

Fusing the observed poses corresponding to the images in each frame to obtain the predicted poses;

The predicted pose of the location marker is marked in the map coordinate system.

In a second possible implementation manner of the embodiment of the present disclosure, the sum of the observed differences between the predicted pose and each of the observed poses is minimized.

In a third possible implementation manner of the embodiment of the present disclosure, at least two of the position identifiers are displayed in each frame of the image;

Wherein, the fusion of the observed poses corresponding to the images of each frame to obtain the predicted poses includes:

Determine the relative observation pose according to the observed pose of each of the position markers corresponding to the same frame of the image;

Determine the predicted poses of the at least two position markers according to the relative observed poses corresponding to the images of each frame; wherein, the relative poses between the predicted poses of the at least two position markers are the same as each frame. The sum of the observed differences between the relative observed poses corresponding to the images is minimized.

In a fourth possible implementation manner of the embodiment of the present disclosure, the observed difference is determined according to an error distance between the predicted pose and each of the observed poses.

In a fifth possible implementation manner of the embodiment of the present disclosure, according to the coordinate system part in the location identifier, decoding the encoding part in the location identifier to obtain the target code includes:

Determine a candidate coordinate system according to the coordinate system part in the position identification;

The target code is obtained by decoding according to the reference coordinate position of the coding part in the position identification in the candidate coordinate system.

In a sixth possible implementation manner of the embodiment of the present disclosure, the coding part includes a plurality of first marking points; the reference coordinate position in the candidate coordinate system according to the coding part in the position identification, Decoding to obtain the target code, including:

performing coordinate system transformation between the candidate coordinate system and the standard coordinate system to obtain an affine transformation matrix between the candidate coordinate system and the standard coordinate system;

Using the affine transformation matrix, transform the coordinate positions of the first marker points in the candidate coordinate system to the standard coordinate system, so as to obtain the standard coordinate system of the first marker points. The coordinate position in ;

According to the coordinate position of each first marker point in the standard coordinate system, the corresponding target code is determined.

In a seventh possible implementation manner of the embodiment of the present disclosure, the coordinate system part includes at least five second marking points;

Wherein, determining the candidate coordinate system according to the coordinate system part in the position identification includes:

In the image, connecting at least three of the second marking points that are collinear to obtain two connecting lines;

Determining the second marked point at the intersection of the two connecting lines as the origin of the candidate coordinate system, and determining the two connecting lines as the coordinate axis of the candidate coordinate system; wherein, the coordinate axis The direction of , is determined according to the distance between the second marker point on the coordinate axis and the origin.

In an eighth possible implementation manner of the embodiment of the present disclosure, the coordinate system part includes an asymmetric pattern;

Determine the coordinate axis of the candidate coordinate system according to the set reference line in the asymmetric pattern; wherein, the direction of the coordinate axis is determined according to the location of the set local pattern in the asymmetric pattern;

And/or, according to the position of the set key point in the asymmetric pattern, the set coordinate point in the candidate coordinate system is determined.

The method for building a map according to the embodiment of the present disclosure obtains the images collected from the mobile device during the moving process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; part, decode the coding part in the position mark to obtain the target code; according to the target code, query to obtain the standard coordinate position of the coding part in the standard coordinate system where the position mark is located; according to the standard coordinate position and the reference coordinate position, determine the standard coordinate system and the standard coordinate position The coordinate transformation relationship between the coordinate systems of the self-mobile device when the image is collected; wherein, the reference coordinate position is the coordinate position of the coding part in the candidate coordinate system determined by the coordinate system part; The device pose in the coordinate system, and the predicted pose of the location marker is marked in the map coordinate system. In the present disclosure, the map is established according to the location identification, even if the display position in the space where the mobile device is located changes, or the movement of people interferes with the lidar scanning the surrounding environment, the established map will not fail, so that the self-mobile device can be effectively located. location, that is, the method is not easily disturbed by the surrounding environment, which can improve the applicability of the method.

The embodiment of the second aspect of the present disclosure provides an apparatus for establishing a map, including:

an acquisition module, configured to acquire an image collected from the mobile device during the movement process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; wherein, the image displays a position identifier;

A decoding module, for decoding the coding part in the position identification to obtain the target code according to the coordinate system part in the position identification;

a query module, configured to query and obtain the standard coordinate position of the coding part in the standard coordinate system where the position marker is located according to the target code;

a determining module, configured to determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image was collected according to the standard coordinate position and the reference coordinate position; wherein the reference coordinate position , is the coordinate position of the coding part in the candidate coordinate system determined by the coordinate system part;

A labeling module, configured to label the predicted pose of the location identifier in the map coordinate system according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system.

Wherein, the labeling module includes:

a determining unit, configured to transform the relationship between the standard coordinate system and the coordinate system of the self-moving device when each frame of the image is collected, and according to the coordinate system of the self-moving device when collecting each frame of the image in the map coordinate system The device pose in the map coordinate system is determined in the map coordinate system to determine the observed pose of the position marker corresponding to each frame of the image;

a fusion unit, configured to fuse the observed poses corresponding to the images in each frame to obtain the predicted poses;

A labeling unit, configured to label the predicted pose of the location identifier in the map coordinate system.

Wherein, the fusion unit is used for:

In a fifth possible implementation manner of the embodiment of the present disclosure, the decoding module includes:

a processing unit, configured to determine a candidate coordinate system according to the coordinate system part in the position identification;

and a decoding unit, configured to decode and obtain the target code according to the reference coordinate position of the encoding part in the position identification in the candidate coordinate system.

In a sixth possible implementation manner of the embodiment of the present disclosure, the encoding part includes a plurality of first marking points; the decoding unit is configured to:

Wherein, the processing unit is used for:

The apparatus for establishing a map according to the embodiment of the present disclosure obtains the images collected from the mobile device during the movement process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; part, decode the coding part in the position mark to obtain the target code; according to the target code, query to obtain the standard coordinate position of the coding part in the standard coordinate system where the position mark is located; according to the standard coordinate position and the reference coordinate position, determine the standard coordinate system and the reference coordinate position. The coordinate transformation relationship between the coordinate systems of the self-mobile device when the image is collected; wherein, the reference coordinate position is the coordinate position of the coding part in the candidate coordinate system determined by the coordinate system part; The device pose in the coordinate system, and the predicted pose of the location marker is marked in the map coordinate system. In the present disclosure, the map is established according to the location identification, even if the display position in the space where the mobile device is located changes, or the movement of people interferes with the lidar scanning the surrounding environment, the established map will not fail, so that the self-mobile device can be effectively located. position to enhance the applicability of the device.

The embodiment of the third aspect of the present disclosure provides a self-mobile device, including: a memory, a processor, and a computer program stored in the memory and running on the processor, when the processor executes the program, the implementation of the The method for building a map proposed by the embodiments of the first aspect is disclosed.

Embodiments of the fourth aspect of the present disclosure provide a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method for establishing a map provided by the embodiments of the first aspect of the present disclosure.

Embodiments of the fifth aspect of the present disclosure provide a computer program product. When instructions in the computer program product are executed by a processor, the method for building a map as proposed by the embodiments of the first aspect of the present disclosure is implemented.

Additional aspects and advantages of the present disclosure will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the present disclosure.

Description of drawings

In order to explain the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a method for establishing a map provided by Embodiment 1 of the present application;

2 is a schematic flowchart of a method for establishing a map according to Embodiment 2 of the present application;

FIG. 3 is a schematic diagram 1 of a position identification in an embodiment of the present application;

FIG. 4 is a schematic diagram 2 of the position identification in the embodiment of the application;

5 is a schematic flowchart of a method for establishing a map provided by Embodiment 3 of the present application;

6 is a schematic flowchart of a method for establishing a map provided by Embodiment 4 of the present application;

7 is a schematic structural diagram of an apparatus for establishing a map according to Embodiment 5 of the present application;

FIG. 8 is a schematic structural diagram of an apparatus for establishing a map according to Embodiment 6 of the present application.

Detailed ways

Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present disclosure and should not be construed as a limitation of the present disclosure.

In the related art, by carrying a laser radar on a self-mobile device, in the process of moving the self-mobile device, the laser radar is used to scan the surrounding environment to obtain a laser point cloud image. The location of the mobile device builds a map. Furthermore, during navigation and positioning, the real-time position of the self-mobile device in the map can be determined by matching the point cloud image collected by the lidar in real time with the pre-established map, so as to realize the positioning of the self-mobile device.

However, the above methods cannot be applied to scenarios where the indoor environment frequently changes. For example, when the position of indoor furnishings such as indoor furniture changes, or when the movement of people interferes with lidar scanning the surrounding environment, the pre-established map will become invalid. The created map will not match to locate the location from the mobile device.

In view of the above problems, the present disclosure proposes a method, apparatus, self-moving device and storage medium for establishing a map.

The method, apparatus, self-moving device, and storage medium for establishing a map according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a method for establishing a map according to Embodiment 1 of the present disclosure.

The executive body of the embodiment of the present disclosure may be the apparatus for establishing a map provided by the present disclosure, and the apparatus for establishing a map may be configured in a self-mobile device. For example, the apparatus for establishing a map may be a local controller of the self-mobile device, so that it can The function of creating a map is implemented by the mobile device; alternatively, the device for creating a map can also be configured in a server, for example, the device for creating a map can be a cloud server that communicates with the self-mobile device, so that the server can create a map. function.

Among them, the self-moving device may be a device that has the function of navigation and obstacle avoidance and can move autonomously, such as an intelligent robot.

As shown in Figure 1, the method for establishing a map may include the following steps:

Step 101 , acquiring an image collected from the mobile device during the moving process, and the device pose of the self-mobile device in the map coordinate system when the image was collected; wherein, the image displays a position identifier.

In this embodiment of the present disclosure, the map coordinate system is the world coordinate system, which refers to a coordinate system generated when a map is created from a mobile device or a server. For example, when creating a map, the position where the mobile device is started can be used as the origin of the map coordinate system, the forward direction of the mobile device can be used as the positive direction of the X axis, the direction perpendicular to the X axis can be used as the Y axis, and the direction perpendicular to the X axis can be used as the Y axis. The direction perpendicular to the X axis and the Y axis can be used as the Z axis, or the forward direction of the self-moving device can be used as the positive direction of the Y axis, the direction perpendicular to the Y axis can be used as the X axis, and the direction perpendicular to the X axis and the Y axis Can be used as Z axis. That is, the origin of the map coordinate system is the position when the mobile device is activated, and each coordinate axis and the direction of each coordinate axis can be predefined.

In the embodiment of the present disclosure, an image sensor may be provided on the self-moving device, and during the movement of the self-moving device, an image may be collected through the image sensor. Among them, the collected images are displayed with location identifiers. The image sensor may be an image sensor such as a Charge Coupled Device (CCD for short), a Complementary Metal Oxide Semiconductor (CMOS for short), and a Thin Film Transistor (TFT for short).

In this embodiment of the present disclosure, when the apparatus for creating a map is configured in a self-mobile device, the apparatus for creating a map can directly acquire an image captured by an image sensor, and when the apparatus for creating a map is configured in a server, the device for creating a map can be configured by the self-mobile device After receiving the image captured by the image sensor, the image is sent to the server, so that the map building apparatus can acquire the image.

In this embodiment of the present disclosure, a two-dimensional location marker (or road sign) may be set in the space where the mobile device is located, for example, the location marker may be attached to an indoor wall or roof. Wherein, a coordinate system part and a coding part can be displayed on the position marker, the coordinate system part is used to determine the coordinate system, and the coding part is used to decode the code corresponding to the position marker and locate the observation position.

It can be understood that the shape, size, color and other characteristics of the coordinate system part and the coding part in the location identification are known. After the image captured by the image sensor is acquired, the coordinate system part and the coding part in the image can be identified based on the target detection algorithm.

For example, the images collected by the image sensor can be recognized based on target detection algorithms such as Single Shot MultiBox Detector (SSD), You Only Look Once (YOLO), Faster-RCNN, etc. , determine the coordinate system part and the encoding part, which is not limited in the present disclosure.

In another possible implementation manner of the embodiment of the present disclosure, after acquiring the image collected by the image sensor, the map building apparatus may perform connected domain detection on the image collected by the image sensor to obtain multiple connected areas, According to the geometric features of each connected region, the coordinate system part and the coding part are determined. The geometric feature may be feature information such as the size, aspect ratio, and color distribution of the connected region.

It should be understood that, in order to improve the accuracy of the recognition results and improve the processing efficiency of the image, the location identifier in the image can also be identified first. Detection algorithm, etc., identify the area where the position marker is located in the image, and then identify the coordinate system part and the coding part in the area where the position marker is located.

In another possible implementation manner of the embodiment of the present disclosure, after acquiring the image collected by the image sensor, the map building apparatus may perform preprocessing on the image collected by the image sensor, such as Gaussian blur, binarization , edge extraction and other processing, according to the value of each pixel in the preprocessed image, determine the coordinate system part and the coding part. Alternatively, the coordinate system part and the encoding part can also be determined directly according to the value of each pixel in the image collected by the image sensor.

For example, each pixel in the image whose value exceeds a preset threshold can be determined, and a connected domain can be determined according to each pixel whose value exceeds the preset threshold, and then the coordinate system can be identified according to the geometric characteristics of each connected domain. section and coding section. That is to say, the identification of the coordinate system part and the coding part can be realized according to the brightness in the connected domain, and the connected domain whose shape is similar to the coordinate system part and the coding part can be excluded, but the brightness does not meet the conditions, so as to improve the coordinate system part and the accuracy of the detection results of the coding part.

In this embodiment of the present disclosure, the device pose may include a coordinate position and/or a pose of the mobile device. The device pose of a self-moving device can be detected by relevant sensors. For example, the device pose of a self-moving device can be measured by sensors such as lidar, odometer, and Inertial Measurement Unit (IMU). For example, after the self-mobile device is started, the map coordinate system can be determined, and the coordinate position of the self-mobile device in the map coordinate system can be determined according to the moving direction and the moving distance of the self-mobile device, and according to the movement of the self-mobile device. The moving direction, moving angle and moving distance of each time in the process can determine the pose of the self-moving device relative to the initial startup, and the pose is the device pose of the self-moving device. The moving angle may be an angle between the moving direction and the reference direction calibrated by the IMU.

In this embodiment of the present disclosure, when the apparatus for establishing a map is configured in a self-moving device, the self-moving device can directly acquire the above-mentioned image and the device pose of the self-moving device in the map coordinate system when the image is captured, and when the self-moving device is established When the map device is configured in the server, after the self-mobile device collects the image during the movement, the collected image and the device pose of the self-mobile device in the map coordinate system when the image is collected can be sent to the server, and the corresponding image is sent to the server. , the server can receive the above image and the device pose of the self-mobile device in the map coordinate system when the image is collected.

Step 102: Decode the coding part in the position identification according to the coordinate system part in the position identification to obtain the target code.

In the embodiment of the present disclosure, a candidate coordinate system can be established according to the coordinate system part in the location identifier, and the target code can be obtained by decoding according to the coordinate position of the coding part in the location identifier in the candidate coordinate system, which is recorded as the reference coordinate position in this disclosure. .

The candidate coordinate system is a two-dimensional coordinate system, and the candidate coordinate system is a coordinate system established on the captured image, for example, a coordinate system constructed from a coordinate system part in the image. Wherein, the unit in the candidate coordinate system may be a pixel, or may also be set according to actual requirements, for example, the unit in the candidate coordinate system may be set as a set length, and the set length may be, for example, 0.001cm, 0.01cm, etc. , this disclosure does not limit this

In the embodiment of the present disclosure, after the candidate coordinate system is established in the image according to the coordinate system part, the reference coordinate position of the coding part in the location identifier in the candidate coordinate system can be determined, and then, the coding part in the location identifier can be used in the candidate coordinate system. The reference coordinate position in the coordinate system is decoded to obtain the target code.

In a possible implementation of the embodiment of the present disclosure, the encoding part may include a plurality of first marking points, and the coordinate position of each first marking point in the candidate coordinate system may be determined, according to the position of each first marking point in the candidate coordinate system The coordinate position in the coordinate system can be decoded to obtain the target code.

It can be understood that the first marker point may contain multiple pixel points, and for each first marker point, the coordinate position of the first marker point in the candidate coordinate system may be based on the number of pixels included in the first marker point. The coordinate position of each pixel is determined.

As an example, for each first marker point, the coordinate positions of the multiple pixel points included in the first marker point in the candidate coordinate system may be determined, and the multiple pixel points included in the first marker point are located in the candidate coordinate system. The coordinate positions in the candidate coordinate system are averaged to determine the coordinate position of the first marker point in the candidate coordinate system.

As another example, for each first marker point, the coordinate positions of multiple pixel points included in the first marker point in the candidate coordinate system may be determined, and according to the multiple pixel points included in the first marker point At the coordinate position in the candidate coordinate system, the shape of the first marked point is fitted with a mathematical equation, and the centroid of the first marked point is determined according to the fitted mathematical equation, and the coordinate position of the centroid is taken as the The coordinate position of the first marker point in the candidate coordinate system.

In a possible implementation of the embodiment of the present disclosure, when the encoding part includes a plurality of first marking points, after determining the coordinate position of each first marking point in the candidate coordinate system, each first marking point may be Map to the standard coordinate system to obtain the coordinate position of each first marker point in the standard coordinate system, and determine the corresponding target code according to the coordinate position of each first marker point in the standard coordinate system.

For example, the number of the first marker points is 3, and the coordinate positions of the 3 first marker points in the standard coordinate system are (1, 1), (2, 2) and (3, 3) respectively, then The target code can be, for example, 112233, 11-22-33, 11 22 33, 1-1-2-2-3-3, and so on.

Step 103 , according to the target code, query to obtain the standard coordinate position of the encoded part in the standard coordinate system where the position identifier is located.

In the embodiment of the present disclosure, the standard coordinate system is the coordinate system where the location marker is located, and the standard coordinate system is a coordinate system pre-established on the location marker according to the coordinate system part on the location marker in the space where the mobile device is located. It should be understood that the image collected by the image sensor may be distorted, and the candidate coordinate system may be distorted or distorted. For the observed coordinate system, it changes with the change of the observation position, and the coordinate axis may not be straight. The standard coordinate system corresponds to the candidate coordinate system, and there is no distorted coordinate system, that is, the standard coordinate system does not change with the change of the observation position.

In the embodiment of the present disclosure, since the candidate coordinate system may be distorted, and the target encoding is determined according to the coordinate position of the encoding part in the candidate coordinate system, the target encoding may deviate from the actual encoding. Therefore, in the present disclosure, in order to improve the accuracy of the positioning result, the standard coordinate position of the encoded part in the standard coordinate system where the position identifier is located can be obtained by querying according to the target code.

It should be understood that each location identifier is known, and the encoding part and the coordinate system part in the location identifier are also known. After the location identifier is set in the space where the mobile device is located, the standard coordinate system can be determined, and the encoding part can be determined. The standard coordinate position in the standard coordinate system can also be determined. For example, when the encoding part includes a plurality of first marking points, the standard coordinate position of each first marking point in the encoding part in the standard coordinate system can be calculated. owned. Therefore, in the present disclosure, for each location identifier, the standard coordinate position of the encoded part in the location identifier in the standard coordinate system where the location identifier is located can be pre-calculated, and each location identifier is stored correspondingly, which is in the same position as the encoded part in the location identifier. The standard coordinate position in the standard coordinate system in which the location ID is located.

Therefore, in the present disclosure, the stored data can be queried according to the target code, and the standard coordinate position matching the target code can be obtained as the standard coordinate position of the code part in the standard coordinate system where the position identifier is located.

For example, when the coding part includes a plurality of first marking points and the number of the first marking points is 3, assuming that the target code is 1-1-2-2-3.1-3.1, the target code can be queried with the target code The standard coordinate positions of the three first marked points in the same position identification with the highest matching degree, such as (1, 1), (2, 2) and (3, 3), can be queried to obtain the standard coordinate positions , as the standard coordinate positions of the three first marking points in the coding part in the standard coordinate system where the position identifier is located.

Step 104, according to the standard coordinate position and the reference coordinate position, determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is collected; wherein, the reference coordinate position is the candidate coordinate determined by the coding part in the coordinate system part coordinate position in the system.

In this embodiment of the present disclosure, the coordinate system of the self-moving device is a pre-calibrated coordinate system. For example, the coordinate system of the self-moving device may be a coordinate system that is pre-calibrated on the self-moving device, such as the coordinate system of the self-moving device. The origin can be the center of mass of the mobile device, the Y-axis is vertically up, and the X-axis is horizontally right or left.

In the embodiment of the present disclosure, after the candidate coordinate system is established in the image, the reference coordinate position of the coding part in the position identification in the candidate coordinate system can be determined. For example, when the coding part includes a plurality of first marking points, according to the content recorded in step 102, the coordinate position of each first marking point in the candidate coordinate system can be determined, and the coordinate position of each first marking point in the candidate coordinate system can be determined. Coordinate position, as the reference coordinate position.

In the embodiment of the present disclosure, the PnP algorithm can be used to determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is captured according to the standard coordinate position and the reference coordinate position corresponding to the encoding part. For example, when the coding part includes a plurality of first marking points, the PnP algorithm can be used to determine the relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is captured according to the standard coordinate position and the reference coordinate position corresponding to the plurality of marking points. Coordinate transformation relationship. The coordinate transformation relationship may include a rotation matrix R and a displacement vector t (or a translation vector) between the standard coordinate system and the coordinate system of the mobile device when the image is captured. For example, the coordinate transformation relationship may be composed of a rotation matrix R and a displacement The transformation matrix T_t composed of the vector t.

Among them, the PnP (pespective-n-point) algorithm may include algorithms such as P3P, EPnP, UPnP, DLT (Direct Linear Transform), and optimization solution.

Step 105 , according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system, mark the predicted pose of the position marker in the map coordinate system.

In the embodiment of the present disclosure, the position identifier in the map coordinate system can be determined according to the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is collected, and the device pose of the self-mobile device in the map coordinate system The predicted pose of , and the predicted pose of the location marker is marked in the map coordinate system.

For example, if the predicted pose of the marker position identifier is Target_id_P, and the device pose of the self-mobile device in the map coordinate system is P, the predicted pose Target_id_P can be:

In the present disclosure, after the map is established, the self-mobile device can be navigated and positioned according to the established map. For example, during positioning, the image collected by the image sensor can be obtained; wherein, the coordinate system part and the coding part are displayed in the image; according to the coordinate system part, the candidate coordinate system is determined; according to the reference coordinate position of the coding part in the candidate coordinate system, Decode to obtain the target code; according to the target code, query to obtain the coordinate position of the coding part in the world coordinate system; according to the coordinate position of the coding part in the world coordinate system and the coordinate position in the image, determine the self-mobile device equipped with the image sensor The pose in the world coordinate system.

As a possible implementation, when the coding part includes multiple first marking points, the target coding can be obtained by decoding according to the coordinate position of each first marking point in the coding part in the candidate coordinate system; according to the target coding, query Obtain the coordinate position of each first marker point in the world coordinate system; according to the coordinate position of each first marker point in the world coordinate system and the coordinate position in the image, determine the self-mobile device equipped with the image sensor in the world coordinate system lower pose. For example, the PnP algorithm can be used to determine the pose of the self-mobile device equipped with the image sensor in the world coordinate system according to the coordinate position of each first marker point in the world coordinate system and the coordinate position in the image.

The coordinate position of each first marker point in the image may be the coordinate position of each first marker point in the image coordinate system, or may also be the coordinate position of each first marker point in the pixel coordinate system. There is no limit to this disclosure. Among them, the coordinate origin of the image coordinate system is the center point of the image, the X axis is horizontal to the right, the Y axis is horizontally downward, and the unit is pixel. The coordinate origin of the pixel coordinate system is the upper left corner of the image, the X axis is horizontal to the right, the Y axis is horizontally downward, and the unit is pixels.

The query process may be as follows: querying the created map according to the target code, obtaining the pose of the location marker to which the target code belongs in the world coordinate system, according to the pose of the location marker in the world coordinate system, and each first marker point in the world coordinate system The coordinate position in the standard coordinate system where the position identifier is located determines the coordinate position of each first marker point in the world coordinate system. The above-mentioned poses may include coordinate positions and/or poses.

It should be noted that when the self-mobile device creates a map, the self-mobile device can store the map locally after the map is created, so that navigation and positioning can be performed according to the map; or, when the server locates the self-mobile device , after the self-mobile device establishes the map, it can send the created map to the server, and after the server receives the map, it can store the corresponding relationship between the identification of the self-mobile device and the map in the database, so that when the When navigating and positioning the self-mobile device, the server may query the above correspondence according to the identification of the self-mobile device, determine the map corresponding to the self-mobile device, and perform navigation and positioning on the self-mobile device according to the queried map.

When a map is created by the server, after creating the map, the server may store the correspondence between the identifiers of the respective mobile devices and the map in the database. Therefore, when the server performs navigation and positioning on the self-mobile device, the server can query the above-mentioned correspondence according to the identification of the self-mobile device, determine the map corresponding to the self-mobile device, and perform navigation and positioning on the self-mobile device according to the inquired map; Alternatively, when the self-moving device locates itself, the server can query the above-mentioned correspondence according to the identification of the self-moving device, determine the map corresponding to the self-moving device, and send the corresponding map to the self-moving device, or the The self-mobile device can actively query its corresponding map from the server side, so that the self-mobile device can perform navigation and positioning according to the acquired map.

It should be noted that there may be more than one solution of the PnP algorithm. In order to improve the accuracy of the pose calculation results of the self-mobile device in the world coordinate system, a check point can be set in the standard coordinate system where the position marker is located. The affine transformation matrix between the candidate coordinate system and the standard coordinate system, determine the coordinate position of the check point in the candidate coordinate system, and use the coordinate position of the check point in the candidate coordinate system to check multiple solutions , to rule out wrong poses.

It should be noted that the execution sequence of positioning and creating a map (that is, building a map) can be executed sequentially, that is, first executing the mapping, and then executing the positioning, or, it can also execute the mapping and positioning at the same time. Figure, after performing the positioning for an example.

It should be noted that, in order to facilitate image recognition, a high-contrast design can be used for the design of the position identification. For example, the background color of the position identification can be black, and the color of the coordinate system part and the coding part can be white. The background color can be white, the color of the coordinate system part and the coding part can be black, or the position identification can also be designed in a highly reflective form, etc., which is not limited in the present disclosure. The color of the coordinate system part and the coding part can be the same or different.

In a possible implementation of the embodiment of the present disclosure, the location identification may be affected by lighting and illumination, and when the ambient brightness is high or low, the accuracy of the image recognition result will be affected. Therefore, in this disclosure, in order to Improve the accuracy of the recognition results of the coordinate system part and the coding part in the collected images, thereby improving the accuracy of the pose calculation results. When a two-dimensional position mark is set in the space where the mobile device is located, the coordinates in the position mark The position of the system part and the coding part can be provided with an infrared LED light source for emitting infrared light to the outside, and the image sensor can be an infrared camera, so that the image containing the position identification can be taken, which can be free from the influence of the mobile device in the space. The influence of ambient brightness can improve the accuracy of the recognition results. In addition, the wavelength of infrared light can be 940 nanometers, so as to be invisible to the user's naked eye, thereby avoiding disturbing the user.

In another possible implementation of the embodiment of the present disclosure, the location marker may be designed in a highly reflective form, and the self-moving device may carry a light emitter for emitting light outward, and the emitted light is projected onto the location marker for reflection. , so that the location marker can be captured according to the light reflected on the location marker.

For example, the light emitter may comprise a light emitting diode, and the light emitting diode may emit visible light, or may also be non-visible light, such as infrared light.

As a possible implementation, when the encoding part includes multiple first markers, the coordinate positions of the first markers in the candidate coordinate system can be mapped to the standard coordinate system where the location identifier is located to obtain the first marker The coordinate position of the point in the standard coordinate system, and the target code is determined according to the coordinate position of each first marker point in the standard coordinate system. The above process will be described in detail below with reference to the second embodiment.

FIG. 2 is a schematic flowchart of a method for establishing a map according to Embodiment 2 of the present disclosure.

As shown in Figure 2, the method for establishing a map may include the following steps:

Step 201, obtain the image collected from the mobile device during the moving process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; wherein, the image is displayed with a position mark, and the position mark includes a coordinate system part and a coding part , the coding part includes a plurality of first marking points.

For the execution process of step 201, reference may be made to the execution process of step 101 in the foregoing embodiment, and details are not described herein.

Step 202: Determine a candidate coordinate system according to the coordinate system part.

In a possible implementation manner of the embodiment of the present disclosure, the coordinate system part may include an asymmetric image, and the coordinate axis of the candidate coordinate system may be determined according to a set reference line in the asymmetric image; wherein, the direction of the coordinate axis, It is determined according to the position of the set local pattern in the asymmetric pattern.

For example, the asymmetric pattern can have two connecting lines, and these two connecting lines can be used as the X-axis and Y-axis of the candidate coordinate system, and the directions of the X-axis and Y-axis can be set according to the local asymmetric pattern. The location of the pattern is determined, for example, the quadrant where the partial pattern is located can be set as the first quadrant, so that the directions of the X-axis and the Y-axis can be determined. Of course, the quadrant in which the partial pattern is located may also be set as the second quadrant, the third quadrant or the fourth quadrant, which is not limited in the present disclosure.

It should be noted that the above is only an example of determining the direction of the coordinate axis according to the location of the set local pattern in the asymmetric pattern. In practical applications, the direction of the coordinate axis can also be determined directly according to the image features of the set local pattern. For example, if the partial pattern is set as an arrow pattern, the positive direction of the coordinate axis can be determined according to the arrow direction of the set partial pattern.

In another possible implementation of the embodiment of the present disclosure, the coordinate system part may include an asymmetric image, and the coordinate point set in the candidate coordinate system may be determined according to the position of the key point set in the asymmetric pattern. For example, it is known that the coordinate point set in the candidate coordinate system of the key point in the known asymmetric pattern is (-1, 1). Candidate coordinate system.

In yet another possible implementation of the embodiment of the present disclosure, the coordinate system part may include an asymmetric image, the coordinate axis of the candidate coordinate system may be determined according to the set reference line in the asymmetric pattern, and the coordinate axis of the candidate coordinate system may be determined according to the set reference line in the asymmetric pattern. Set the position of the key point, and determine the coordinate point set in the candidate coordinate system. Wherein, the direction of the coordinate axis is determined according to the position where the local pattern is set in the asymmetric pattern.

As an example, refer to FIG. 3 , which is a schematic diagram 1 of a location identification in an embodiment of the present disclosure. Wherein, the position identification includes a set partial pattern 21 and an encoding part 22 composed of a plurality of marking points (referred to as first marking points in the present disclosure). The setting reference line can be the symmetry axis of the setting partial pattern 21, and the setting reference line can be used as the X axis in the candidate coordinate system, and the arrow direction of the setting partial pattern 21 can be regarded as the positive direction of the X axis, and the asymmetrical At least one set key point is set in the pattern. The set key point can be the origin of the candidate coordinate system, or a point in the positive or negative direction of the Y axis. According to the set key point and the X axis, you can Create a candidate coordinate system.

In yet another possible implementation of the embodiment of the present disclosure, the coordinate system part may include at least five second marking points, and in the image, at least three second marking points that are collinear may be connected to obtain two connecting lines , the second mark point at the intersection of the two connecting lines is determined as the origin of the candidate coordinate system, and the two connecting lines are determined as the coordinate axis of the candidate coordinate system; wherein, the direction of the coordinate axis is based on the coordinate axis. The distance between the second marker point and the origin is determined.

As an example, refer to FIG. 4 , which is a second schematic diagram of position identification in an embodiment of the present disclosure, wherein letter A represents the second marking point, and letter B represents the first marking point. The distance between each second marker point and the origin can be determined, the positive direction of the coordinate axis is determined according to the side with the longer distance, and the negative direction of the coordinate axis is determined according to the side with the short distance. It should be noted that Fig. 4 only uses the positive direction of the coordinate axis according to the side with the long distance, and the negative direction of the coordinate axis according to the side with the short distance as an example. The negative direction of the coordinate axis is determined, and the positive direction of the coordinate axis is determined according to the side with the short distance, which is not limited in the present disclosure. For convenience of description, the present disclosure takes as an example that the positive direction of the coordinate axis is determined according to the side with a long distance, and the negative direction of the coordinate axis is determined according to the side with a short distance.

It should be noted that when the shape, size, color and other characteristics of the plurality of second marking points in the coordinate system part and the plurality of first marking points in the coding part are exactly the same, based on the image recognition technology in step 101, only the After identifying each marker point in the image, at this time, it is necessary to further determine a plurality of first marker points and a plurality of second marker points from each marker point. For example, referring to FIG. 4 , the image features of the plurality of first marking points B and the plurality of second marking points A are the same.

As a possible implementation manner of the embodiment of the present disclosure, in order to improve the accuracy of the recognition result, each marker point may be first identified from the image, and then each second marker point may be identified from each marker point; The position distribution of the marked points complies with the set geometric constraints, so that each marked point other than the second marked point can be used as the first marked point.

For example, the geometric constraints set above may be asymmetric geometric constraints. When the position distribution of the second marker points complies with the asymmetric geometric constraints, the geometric constraints can be used to determine the position of the candidate coordinate system in the image. The axis and the direction of the axis.

It should be noted that the

above steps

101 and 202 are only exemplary embodiments, but the present disclosure is not limited thereto, and other image recognition methods known in the art may also be included, as long as each marker point in the image can be recognized. . For example, the image collected by the image sensor can also be preprocessed, such as Gaussian blur, binarization, edge extraction, etc., and each marker point can be determined according to the value of each pixel in the preprocessed image, or, Each marker point can be determined directly according to the value of each pixel point in the image collected by the image sensor. For example, each pixel in the image whose value exceeds a preset threshold may be determined, and a connected domain may be determined according to each pixel whose value exceeds the preset threshold, and each connected domain may be used as each marked point. Alternatively, each pixel point in the image whose value exceeds the preset threshold may be directly used as each marker point, which is not limited in the present disclosure.

That is to say, each marker point can be identified according to the brightness in the connected domain, and the connected domain whose shape is similar to the marker point but whose brightness does not satisfy the condition can be excluded, so as to improve the accuracy of the marker point detection result.

It should be understood that, in the above-mentioned embodiment, when the position distribution of the second marker points complies with the asymmetric geometric constraints, the direction of the coordinate axis in the candidate coordinate system can be determined, and when the coordinate axis has a direction, it is encoded in different quadrants. With different meanings, this way can also increase the encoding capacity.

Step 203: Perform coordinate system transformation between the candidate coordinate system and the standard coordinate system to obtain an affine transformation matrix between the candidate coordinate system and the standard coordinate system.

The standard coordinate system is the coordinate system where the location marker is located, that is, the standard coordinate system is a coordinate system pre-established on the location marker according to the part of the coordinate system on the location marker in the space where the mobile device is located. It should be understood that the image collected by the image sensor may be distorted, and the candidate coordinate system may be distorted or distorted. For the observed coordinate system, it changes with the change of the observation position, and the coordinate axis may not be straight. The standard coordinate system corresponds to the candidate coordinate system, and there is no distorted coordinate system, that is, the standard coordinate system does not change with the change of the observation position.

In this embodiment of the present disclosure, affine transformation can be understood as a new coordinate axis formed by scaling, rotating, and translating the original coordinate axis. After the candidate coordinate system and the standard coordinate system are determined, the affine transformation matrix between the candidate coordinate system and the standard coordinate system can be determined according to the two-dimensional geometric transformation.

Step 204, using an affine transformation matrix to transform the coordinate position of each first marker point in the candidate coordinate system to the standard coordinate system, so as to obtain the coordinate position of each first marker point in the standard coordinate system.

In the embodiment of the present disclosure, for each first marker point in the candidate coordinate system, an affine transformation matrix may be used to transform the coordinate position of the first marker point into the standard coordinate system, so as to obtain the first marker point in the standard coordinate system. The coordinate position in the coordinate system.

Step 205: Determine the corresponding target code according to the coordinate position of each first marker point in the standard coordinate system.

In the embodiment of the present disclosure, the corresponding target code may be determined according to the coordinate position of each first marker point in the standard coordinate system. For example, the coordinate positions of the first marker points in the standard coordinate system can be combined to obtain the corresponding target code. For example, when the number of the first marker points is 3, it is assumed that the coordinate positions of the 3 first marker points in the standard coordinate system are (1, 1), (2, 2) and (3, 3 respectively) ), the target code can be, for example, 112233, 11-22-33, 11 22 33, 1-1-2-2-3-3, etc.

Step 206, according to the target code, query to obtain the coordinate position of each first marker point in the code part in the standard coordinate system where the position identifier is located, which is recorded as the standard coordinate position in the present disclosure.

It should be understood that each location identifier is known, and the encoding part and the coordinate system part in the location identifier are also known. After the location identifier is set in the space where the mobile device is located, the standard coordinate system can be determined, and the encoding part can be determined. The standard coordinate position of each first marker point in the standard coordinate system can also be determined. For example, referring to FIG. 4, the standard coordinate position of each first marker point in the coding part in the standard coordinate system can be calculated. . Therefore, in the present disclosure, for each location identifier, the standard coordinate position of each first marker point in the coding part of the location identifier in the standard coordinate system where the location identifier is located can be pre-calculated, and each location identifier is stored correspondingly, which is consistent with the location identifier. The standard coordinate position of the first marked point in the coding part of the location identifier in the standard coordinate system where the location identifier is located.

Therefore, in the present disclosure, the stored data can be queried according to the target code, and the standard coordinate position matching the target code can be obtained as the standard coordinate position of each first marker point in the coding part in the standard coordinate system where the position identifier is located.

For example, when the number of the first marking points is 3, assuming that the target code is 1-1-2-2-3.1-3.1, it is possible to query the 3 points in the same location identifier that have the highest matching degree with the target code. The standard coordinate position of the first marker point, such as (1,1), (2,2) and (3,3), the standard coordinate position obtained by the query can be used as the three first marker points in the coding part The standard coordinate position in the standard coordinate system in which the location marker is located.

Step 207, according to the standard coordinate position and the reference coordinate position, determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is collected; wherein, the reference coordinate position is that each first mark point in the coding part is in the candidate position. The coordinate position in the coordinate system.

In this embodiment of the present disclosure, the PnP algorithm may be used to determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is captured, according to the standard coordinate positions and reference coordinate positions corresponding to the plurality of first marker points. The transformation relationship can include a rotation matrix R and a displacement vector t (or a translation vector) between the standard coordinate system and the coordinate system of the mobile device when the image is captured. For example, the coordinate transformation relationship can be composed of a rotation matrix R and a displacement vector t. Composition of transformation matrix T_t.

Step 208 , according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system, mark the predicted pose of the position marker in the map coordinate system.

For the execution process of step 208, reference may be made to the execution process of the foregoing embodiment, and details are not described herein.

It should be noted that, during the movement of the self-moving device, the image sensor can continuously collect images. When different frames of images are collected, the device pose of the self-moving device in the map coordinate system may be different, so the pose calculated according to step 105 can be different, therefore, as a possible implementation manner of the embodiment of the present disclosure, in order to improve the reliability of the calculation result, the pose corresponding to each frame image obtained by the calculation may be fused to obtain the predicted pose of the position identifier, And mark the predicted location of the location marker in the map coordinate system. The above process will be described in detail below with reference to the third embodiment.

FIG. 5 is a schematic flowchart of a method for establishing a map according to Embodiment 3 of the present disclosure.

As shown in FIG. 5 , when there are at least two frames of images showing the same position identification, and the self-mobile device has a corresponding device pose in the map coordinate system when each frame of image is collected, the implementation as shown in FIG. 1 or FIG. 2 is performed. On the basis of example, step 105 or 208 may include the following steps:

Step 301, according to the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when collecting each frame of image, and according to the device pose of the self-mobile device in the map coordinate system when collecting each frame of image, in the map coordinate system. Determine the observation pose of the position marker corresponding to each frame of images.

In the embodiment of the present disclosure, for each frame of image collected, the observed pose of the position identifier corresponding to the frame of image may be calculated according to formula (1).

For example, the image collected at time t0 is marked as M0, and the device pose when M0 is collected from the mobile device in the map coordinate system is P0. The coordinate transformation relationship between the coordinate systems is T_t0. According to formula (1), the observed pose of the position marker corresponding to M0 can be calculated as

In the same way, the image collected at time t1 is marked as M1, and the device pose when M1 is collected from the mobile device in the map coordinate system is P1. The coordinate transformation relationship between the coordinate systems is T_t1. According to formula (1), the observed pose of the position identifier corresponding to M1 can be calculated as

Step 302 , fuse the observed poses corresponding to each frame of images to obtain the predicted poses.

In the embodiment of the present disclosure, after the observed poses of the position identifiers corresponding to each frame of images are calculated and obtained, the observed poses corresponding to each of the frames of images can be fused to obtain the predicted poses.

In a possible implementation manner of the embodiment of the present disclosure, in order to improve the reliability of the predicted pose obtained after fusion, the sum of the observed differences between the predicted pose and each observed pose is minimized.

For example, the predicted pose is Target_id_P, and the observed pose corresponding to each frame of image is

Then, when fusing the observed poses corresponding to each frame of images to obtain the predicted poses, the following formula (2) needs to be satisfied:

Among them, argmin is the function that makes the functional function take the minimum value, "-" indicates the algorithm used to characterize the observation difference, not limited to direct subtraction, the superscript 2 after || represents the square, and the subscript 2 represents the norm 2.

It should be noted that the pose may include coordinate positions and attitudes, which are multiple dimensional values. The above formula (2) only takes the observation difference as the multi-dimensional pose difference between the predicted pose and each of the observed poses for example. , in practical application, the observation difference can also be calculated according to algorithms such as reprojection and point-to-point spacing, which is not limited in the present disclosure. For example, the predicted pose can be projected onto a certain plane, the predicted predicted value after projection can be determined, and each observed pose can be projected onto the plane to determine each observed value after projection, according to the difference between the predicted predicted value after projection and each observed value The difference between the two to determine the observed difference.

That is, in the present disclosure, the observation difference is determined according to the error distance between the predicted pose and the observed pose corresponding to each frame of images, where the error distance may be Euclidean distance, Mahalanobis distance, or the like.

Step 303: Mark the predicted pose of the position marker in the map coordinate system.

In the embodiment of the present disclosure, the predicted pose of the location marker in the map coordinate system is determined, and the predicted pose of the location marker is marked in the map coordinate system.

In the method for building a map according to the embodiment of the present disclosure, when the images showing the same position identification are at least two frames, when the self-mobile device has a corresponding device pose in the map coordinate system when collecting each frame of images, the image is based on the standard coordinate system. The coordinate transformation relationship between the self-mobile device and the coordinate system of the self-mobile device when each frame of image is collected, and the device pose of the self-mobile device in the map coordinate system when each frame of image is collected. Determine the corresponding frame of image in the map coordinate system. The observed pose of the location marker; the observed pose corresponding to each frame image is fused to obtain the predicted pose, and the predicted pose of the location marker is marked in the map coordinate system. As a result, the reliability of the predicted pose calculation result can be improved, thereby improving the accuracy of subsequent positioning.

It should be noted that at least one position marker can be displayed in each frame of image, and when one position marker is displayed in each frame of image, the observed pose of the position marker in each frame of image can be calculated according to step 301 in the above-mentioned embodiment, and The observed pose is fused to obtain the predicted pose. The predicted pose satisfies the constraints of formula (2), and when there are multiple position identifiers displayed in each frame of image, each frame of image can be calculated according to step 301. The observation poses of the position markers are determined, and the relative observation poses are determined according to the observation poses of each position marker corresponding to the same frame of images. After that, at least two position markers can be determined according to the relative observation poses corresponding to each frame image. The predicted pose; wherein, the sum of the relative poses between the predicted poses of at least two position markers and the observed difference between the relative observed poses corresponding to each frame image is minimized, so as to improve the accuracy of the predicted pose calculation results. reliability, thereby improving the accuracy of subsequent positioning.

The above process will be described in detail below with reference to the fourth embodiment.

FIG. 6 is a schematic flowchart of a method for establishing a map according to Embodiment 4 of the present disclosure.

As shown in FIG. 6 , when each frame of image displays at least two position identifiers, on the basis of the embodiment shown in FIG. 5 , step 302 may include the following steps:

Step 401: Determine the relative observation pose according to the observation pose of each position marker corresponding to the same frame of image.

In the embodiment of the present disclosure, for each position marker in the same frame of image, the candidate coordinate system and the standard coordinate system corresponding to different position markers are different, the target codes obtained by decoding are also different, and the coordinate transformation relations obtained by calculation are also different. , so the calculated observation poses are also different, that is, each position marker in the same frame image has a corresponding observation pose, and the relative observation position can be determined according to the observation pose of each position marker corresponding to the same frame image. posture.

For example, when there are two location identifiers in the same frame of image, they are location identifier 1 and location identifier 2 respectively. The image collected at time t0 is marked as M0, and the device pose when M0 is collected from the mobile device in the map coordinate system is P0. According to the position identifier 1 in M0, the standard coordinate system and the coordinates of the self-mobile device when M0 are collected are calculated. The coordinate transformation relationship between the systems is T_t01. According to formula (1), the observed pose of the position marker 1 corresponding to M0 can be calculated as

make

According to the position identifier 2 in M0, the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when collecting M0 is calculated as T_t02, and according to formula (1), the observation position of the position identifier 2 corresponding to M0 can be calculated and obtained pose

make

Then the relative observation pose between the position identification 1 and the position identification 2 is Target_1_t0–Target_2_t0.

For another example, the image collected at time t1 is marked as M1, and the device pose when M1 is collected from the mobile device in the map coordinate system is P1. The coordinate transformation relationship between the coordinate systems of the device is T_t11. According to formula (1), the observed pose of the position marker 1 corresponding to M1 can be calculated as:

make

According to the position identification 2 in M1, the coordinate transformation relationship between the standard coordinate system and the coordinate system of the mobile device when M1 is collected is calculated as T_t12. According to formula (1), the observation position of the position identification 2 corresponding to M1 can be calculated and obtained. pose

make

Then the relative observation pose between the location identifier 1 and the location identifier 2 is Target_1_t1–Target_2_t1.

Therefore, in the present disclosure, the observed pose corresponding to the position identifier 1 corresponding to each frame of images can be marked as Target_1_tn, and the observed pose corresponding to the position identifier 2 is Target_2_tn, then the distance between the position identifier 1 and the position identifier 2 corresponding to each frame image is The relative observation pose is Target_1_tn–Target_2_tn.

Step 402: Determine the predicted poses of at least two position markers according to the relative observation poses corresponding to each frame of images; wherein, the relative poses between the predicted poses of the at least two position markers and the relative observation corresponding to each frame image The sum of observed differences between poses is minimized.

Still taking the above example as an example, the predicted pose corresponding to the marker location identifier 1 is Target_1, and the predicted pose corresponding to the location identifier 2 is Target_2, then the relative pose between the predicted poses of the two location markers is Target_1–Target_2, then The observation difference between the relative pose and the relative observation pose can satisfy the following formula (3):

SUM(argmin||(Target_1–Target_2)–(Target_1_tn–Target_2_tn)|| ² ₂ ); (3)

Among them, the observation difference is determined according to the error distance between the predicted pose and each observed pose.

In the embodiment of the present disclosure, the predicted poses of at least two position markers are determined by the relative observed poses corresponding to each frame of images; wherein the relative poses between the predicted poses of the at least two position markers are related to the The sum of the observation differences between the relative observation poses corresponding to the images is minimized, and a more reliable predicted pose can be obtained, thereby improving the accuracy of the subsequent positioning results.

In order to realize the above embodiments, the present disclosure also provides an apparatus for establishing a map.

FIG. 7 is a schematic structural diagram of an apparatus for establishing a map according to Embodiment 5 of the present disclosure.

As shown in FIG. 7 , the apparatus 100 for establishing a map may include: an acquisition module 110 , a decoding module 120 , a query module 130 , a determination module 140 , and a labeling module 150 .

Wherein, the acquisition module 110 is used for acquiring the image collected from the mobile device during the movement process, and the device pose in the map coordinate system of the self-mobile device when the image is collected; wherein, the image displays the position identification.

The decoding module 120 is configured to decode the coding part in the position identification according to the coordinate system part in the position identification to obtain the target code.

The query module 130 is configured to query and obtain the standard coordinate position of the encoded part in the standard coordinate system where the position identifier is located according to the target code.

The determination module 140 is used to determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when collecting images according to the standard coordinate position and the reference coordinate position; wherein, the reference coordinate position is determined by the coding part in the coordinate system part The coordinate position in the candidate coordinate system of .

The labeling module 150 is configured to label the predicted pose of the position marker in the map coordinate system according to the coordinate transformation relationship and the device pose of the mobile device in the map coordinate system.

Further, in a possible implementation manner of the embodiment of the present disclosure, the images showing the same position identification are at least two frames, and the self-mobile device has a corresponding device pose in the map coordinate system when collecting each frame of images, then Referring to FIG. 8, based on the embodiment shown in FIG. 7, the labeling module 150 may include:

The determining unit 151 is configured to, according to the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when collecting each frame of image, and according to the device pose of the self-mobile device in the map coordinate system when collecting each frame of image, in the In the map coordinate system, determine the observation pose of the position identification corresponding to each frame of image;

The fusion unit 152 is used to fuse the observed poses corresponding to each frame of images to obtain the predicted poses;

The labeling unit 153 is configured to label the predicted pose of the position marker in the map coordinate system.

As a possible implementation, the sum of the observed differences between the predicted pose and each observed pose is minimized.

Further, in a possible implementation manner of the embodiment of the present disclosure, each frame of image displays at least two position identifiers; wherein, the fusion unit 152 is configured to: according to each position identifier corresponding to the same frame of image Observe the pose, and determine the relative observation pose; according to the relative observation pose corresponding to each frame of images, determine the predicted pose of at least two position markers; wherein, the relative pose between the predicted poses of the at least two position markers is the same as The sum of observation differences between the relative observation poses corresponding to each frame image is minimized.

As a possible implementation, the observed difference is determined according to the error distance between the predicted pose and each observed pose.

Further, in a possible implementation manner of the embodiment of the present disclosure, referring to FIG. 8 , on the basis of the embodiment shown in FIG. 7 , the decoding module 120 may include:

The processing unit 121 is configured to determine a candidate coordinate system according to the coordinate system part in the position identification.

The decoding unit 122 is configured to decode and obtain the target code according to the reference coordinate position of the coding part in the position identification in the candidate coordinate system.

In a possible implementation manner of the embodiment of the present disclosure, the encoding part includes a plurality of first marking points; the decoding unit 122 is configured to: perform coordinate system transformation between the candidate coordinate system and the standard coordinate system to obtain the candidate coordinate system and the standard coordinate system. Affine transformation matrix between standard coordinate systems; using the affine transformation matrix, the coordinate position of each first marker point in the candidate coordinate system is transformed to the standard coordinate system, so as to obtain each first marker point in the standard coordinate system According to the coordinate position of each first marker point in the standard coordinate system, the corresponding target code is determined.

In a possible implementation of the embodiment of the present disclosure, the coordinate system part includes at least five second mark points; wherein, the processing unit 121 is configured to: in the image, connect at least three collinear second mark points , two connecting lines are obtained; the second marked point at the intersection of the two connecting lines is determined as the origin of the candidate coordinate system, and the two connecting lines are determined as the coordinate axis of the candidate coordinate system; among them, the direction of the coordinate axis, It is determined according to the distance between the second marker point on the coordinate axis and the origin.

In another possible implementation of the embodiment of the present disclosure, the coordinate system part includes an asymmetric pattern; wherein, the processing unit 121 is configured to: determine the coordinate axis of the candidate coordinate system according to the set reference line in the asymmetric pattern wherein, the direction of the coordinate axis is determined according to the location of the set local pattern in the asymmetric pattern; and/or, the coordinate point set in the candidate coordinate system is determined according to the location of the key point set in the asymmetric pattern.

It should be noted that, the foregoing explanations on the method embodiment for establishing a map are also applicable to the apparatus for establishing a map in this embodiment, and details are not repeated here.

In order to implement the above embodiments, the present disclosure also proposes a self-mobile device, including: a memory, a processor, and a computer program stored in the memory and running on the processor, when the processor executes the program, the The method for establishing a map proposed by any of the foregoing embodiments of the present disclosure.

Embodiments of the fourth aspect of the present disclosure provide a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method for building a map as proposed in any of the foregoing embodiments of the present disclosure.

Embodiments of the fifth aspect of the present disclosure provide a computer program product. When instructions in the computer program product are executed by a processor, the method for building a map as proposed in any of the foregoing embodiments of the present disclosure is implemented.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structures, materials, or features are included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present disclosure pertain.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program is stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present disclosure have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limitations of the present disclosure, and those of ordinary skill in the art may interpret the above-described embodiments within the scope of the present disclosure. Embodiments are subject to variations, modifications, substitutions and variations.

Claims

A method for establishing a map, the method comprising:

Obtaining an image collected from a mobile device during the movement process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; wherein, the image displays a position identifier;

According to the coordinate system part in the position identification, decode the coding part in the position identification to obtain the target code;

According to the target code, query to obtain the standard coordinate position of the code part in the standard coordinate system where the position identifier is located;

According to the standard coordinate position and the reference coordinate position, determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is captured; wherein, the reference coordinate position is the code the coordinate position of the part in the candidate coordinate system determined by the coordinate system part;

According to the coordinate transformation relationship and the device pose of the self-moving device in the map coordinate system, the predicted pose of the location identifier is marked in the map coordinate system.
The method for building a map according to claim 1, wherein the images showing the same location identifier are at least two frames, and the self-mobile device has a corresponding image in the map coordinate system when collecting each frame of images. equipment pose;

Wherein, according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system, marking the predicted pose of the location identifier in the map coordinate system includes:

According to the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when each frame of the image is collected, and according to the device pose of the self-mobile device in the map coordinate system when each frame of image is collected , in the map coordinate system, determine the observation pose of the position marker corresponding to each frame of the image;

Fusing the observed poses corresponding to the images in each frame to obtain the predicted poses;

The predicted pose of the location marker is marked in the map coordinate system.
The method for building a map according to claim 2, wherein the sum of the observed differences between the predicted pose and each of the observed poses is minimized.
The method for building a map according to claim 2 or 3, wherein at least two of the location identifiers are displayed in each frame of the image;

Wherein, the fusion of the observed poses corresponding to the images of each frame to obtain the predicted poses includes:

Determine the relative observation pose according to the observed pose of each of the position markers corresponding to the same frame of the image;

Determine the predicted poses of the at least two position markers according to the relative observed poses corresponding to the images of each frame; wherein, the relative poses between the predicted poses of the at least two position markers are the same as each frame. The sum of the observed differences between the relative observed poses corresponding to the images is minimized.
The method for building a map according to claim 3 or 4, wherein the observed difference is determined according to an error distance between the predicted pose and each of the observed poses.
The method for building a map according to any one of claims 1-5, wherein, according to the coordinate system part in the position identification, decoding the coding part in the position identification to obtain the target code, comprising:

Determine a candidate coordinate system according to the coordinate system part in the position identification;

The target code is obtained by decoding according to the reference coordinate position of the coding part in the position identification in the candidate coordinate system.
The method for building a map according to claim 6, wherein the coding part comprises a plurality of first marking points; the reference coordinate position in the candidate coordinate system according to the coding part in the position identification , decoding to obtain the target code, including:

performing coordinate system transformation between the candidate coordinate system and the standard coordinate system to obtain an affine transformation matrix between the candidate coordinate system and the standard coordinate system;

Using the affine transformation matrix, transform the coordinate positions of the first marker points in the candidate coordinate system to the standard coordinate system, so as to obtain the standard coordinate system of the first marker points. The coordinate position in ;

According to the coordinate position of each first marker point in the standard coordinate system, the corresponding target code is determined.
The method for building a map according to claim 6 or 7, wherein the coordinate system part includes at least five second marking points;

Wherein, determining the candidate coordinate system according to the coordinate system part in the position identification includes:

In the image, connecting at least three of the second marking points that are collinear to obtain two connecting lines;

Determining the second marked point at the intersection of the two connecting lines as the origin of the candidate coordinate system, and determining the two connecting lines as the coordinate axis of the candidate coordinate system; wherein, the coordinate axis The direction of , is determined according to the distance between the second marker point on the coordinate axis and the origin.
The method for establishing a map according to claim 6 or 7, wherein the coordinate system part comprises an asymmetric pattern;

Wherein, determining the candidate coordinate system according to the coordinate system part in the position identification includes:

Determine the coordinate axis of the candidate coordinate system according to the set reference line in the asymmetric pattern; wherein, the direction of the coordinate axis is determined according to the location of the set local pattern in the asymmetric pattern;

And/or, according to the position of the set key point in the asymmetric pattern, the set coordinate point in the candidate coordinate system is determined.
A device for establishing a map, comprising:

an acquisition module, configured to acquire an image collected from the mobile device during the movement process, and the device pose of the self-mobile device in the map coordinate system when the image is collected; wherein, the image displays a position identifier;

A decoding module for decoding the coding part in the position identification to obtain the target code according to the coordinate system part in the position identification;

a query module, configured to query and obtain the standard coordinate position of the coding part in the standard coordinate system where the position identifier is located according to the target code;

A determination module, configured to determine the coordinate transformation relationship between the standard coordinate system and the coordinate system of the self-mobile device when the image is collected according to the standard coordinate position and the reference coordinate position; wherein the reference coordinate position , is the coordinate position of the coding part in the candidate coordinate system determined by the coordinate system part;

A labeling module, configured to label the predicted pose of the location identifier in the map coordinate system according to the coordinate transformation relationship and the device pose of the self-mobile device in the map coordinate system.
A self-moving device is characterized in that, comprising a memory, a processor and a computer program stored in the memory and running on the processor, when the processor executes the program, it realizes any one of claims 1-9. 1. The described method of building a map.
A non-transitory computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the method for establishing a map according to any one of claims 1-9 is implemented.
A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the method for establishing a map according to any one of claims 1-9 is implemented.