WO2020024234A1

WO2020024234A1 - Route navigation method, related device, and computer readable storage medium

Info

Publication number: WO2020024234A1
Application number: PCT/CN2018/098384
Authority: WO
Inventors: 林义闽; 廉士国; 李业
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2018-08-02
Filing date: 2018-08-02
Publication date: 2020-02-06
Also published as: CN109074668B; CN109074668A

Abstract

A Route navigation method, a related device, and a computer readable storage medium, relating to the technical field of computer vision. The method is applied to a terminal or a cloud, and comprises the following steps: obtaining a depth map (101); constructing a three-dimensional point cloud in a world coordinate system according to the depth map, and performing road surface detection according to the three-dimensional point cloud in the world coordinate system to determine first path obstacle information (102); performing image learning according to the depth map to determine second path obstacle information (103); combining the first path obstacle information and the second path obstacle information to obtain third path obstacle information (104); and performing route navigation according to the third path obstacle information (105). By determining the third path obstacle information which better matches an actual road status by combining the first path obstacle information determined by performing road surface detection on the obtained depth map and the second path obstacle information determined by performing image learning according to the depth map, and performing navigation according to the third path obstacle information, the navigation result is more accurate.

Description

Path navigation method, related device and computer-readable storage medium

Technical field

The present application relates to the field of computer vision technology, and in particular, to a path navigation method, a related device, and a computer-readable storage medium.

Background technique

With the development of autonomous driving and navigation technology, current smart devices such as drones and autonomous robots can collect image information through visual depth sensors and analyze based on the acquired image information to achieve obstacle avoidance and navigation.

technical problem

The inventor discovered during the research of the prior art that in the prior art, intelligent devices such as autonomous robots perform autonomous walking, and when navigation is achieved by obtaining obstacle information, the obstacle information on the path is often determined through the positioning results based on the collected visual information, Navigation, but due to the limitation of perspective and the defects of its own equipment, using this method alone may not obtain accurate access obstacle information, so when navigating according to the obtained access obstacle information, the accuracy of navigation is affected.

Technical solutions

A technical problem to be solved in some embodiments of the present application is to provide a path navigation method, a related device, and a computer-readable storage medium to solve the above technical problems.

An embodiment of the present application provides a path navigation method, a related device, and a computer-readable storage medium, including: obtaining a depth map; constructing a three-dimensional point cloud in the world coordinate system according to the depth map, and according to the three-dimensional point cloud in the world coordinate system. The point cloud is used for road surface detection to determine the first path obstacle information; the image learning is used to determine the second path obstacle information according to the depth map; the first path obstacle information and the second path obstacle information are combined to obtain the third path obstacle information ; Perform path navigation based on the third path obstacle information.

An embodiment of the present application further provides a path navigation device. The path navigation device includes: an acquisition module for acquiring a depth map; a first determination module for constructing a three-dimensional point cloud in the world coordinate system according to the depth map, and according to Pavement detection of the three-dimensional point cloud in the world coordinate system to determine the obstacle information of the first road; a second determination module for image learning to determine the obstacle information of the second channel according to the depth map; a merge module for the obstacle of the first channel The object information and the second path obstacle information are combined to obtain the third path obstacle information; the navigation module is configured to perform route navigation according to the third path obstacle information.

An embodiment of the present application further provides an electronic device including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are processed by at least one The processor executes to enable at least one processor to execute the path navigation method involved in any method embodiment of the present application.

The embodiment of the present application further provides a computer-readable storage medium storing computer instructions, and the computer instructions are used to cause a computer to execute the path navigation method involved in any method embodiment of the present application.

Beneficial effect

Compared with the prior art, the embodiment of the present application determines the first path obstacle information determined by road detection based on the acquired depth map and the second path obstacle information determined by image learning based on the acquired depth map to determine The third path obstacle information that more closely matches the actual road conditions, and the path navigation is performed based on the third path obstacle information that is more in line with the actual conditions to improve the accuracy of the path navigation.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments are exemplified by the pictures in the accompanying drawings. These exemplary descriptions do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the drawings in the drawings do not constitute a limitation on scale.

FIG. 1 is a flowchart of a path navigation method in a first embodiment of the present application;

2 is a relationship diagram between a pixel coordinate system and a camera coordinate system in the first embodiment of the present application;

3 is a relationship diagram between a camera coordinate system and a world coordinate system in the first embodiment of the present application;

4 is a flowchart of a path navigation method in a second embodiment of the present application;

5 is a schematic block diagram of a path navigation device in a third embodiment of the present application;

6 is a schematic block diagram of a path navigation device in a fourth embodiment of the present application;

FIG. 7 is a structural example diagram of an electronic device in a fifth embodiment of the present application.

Embodiments of the invention

In order to make the purpose, technical solution, and advantages of the present application clearer, some embodiments of the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

The first embodiment of the present application relates to a route navigation method, which is applied to a terminal or a cloud. The terminal can be a device such as a blind guide helmet, an unmanned vehicle, or an intelligent robot. The cloud communicates with the terminal, providing the terminal with a map for positioning or providing positioning results directly to the terminal. This embodiment uses a terminal as an example to describe the execution process of the path navigation method. For the process of executing the path navigation method in the cloud, reference may be made to the content of the embodiment of the present application.

The specific flow of the route navigation method is shown in Figure 1, and includes the following steps:

In step 101, a depth map is obtained.

Specifically, in this embodiment, the depth map includes a color map and a depth value corresponding to the color map. The way to obtain the depth map may be obtained by the terminal real-time shooting through the camera device during the movement.

In step 102, a three-dimensional point cloud in the world coordinate system is constructed according to the depth map, and road surface detection is performed according to the three-dimensional point cloud in the world coordinate system to determine the first path obstacle information.

Specifically, a three-dimensional point cloud in the first camera coordinate system is constructed according to the depth map, and the pose information of the terminal is obtained. A three-dimensional point cloud in the world coordinate system is constructed according to the three-dimensional point cloud and pose information in the first camera coordinate system. , And then perform road detection according to the three-dimensional point cloud in the world coordinate system to determine the obstacle information of the first path.

In a specific implementation, formula (1) is used to construct a three-dimensional point cloud in the depth camera coordinate system. Formula (1) is expressed as follows:

Among them, u and v are the position coordinates of the pixels in the depth map, f is the internal parameter of the depth camera, X _d , Y _d and Z _d are the coordinate values of the 3D point cloud in the depth camera coordinate system, and H is the depth map. Width, W represents the height of the depth map, and Z _d is the depth value of the pixels in the depth map, which is a known quantity.

In addition, it is necessary to obtain the 3D point cloud of the 3D point cloud in the first camera (color camera) coordinate system according to the conversion relationship between the depth camera and the color camera, and use formula (2) to construct the 3D point cloud in the first camera coordinate system. , Formula (2) is expressed as follows:

[X _c Y _c Z _c ] = [R, T] _cd [X _d Y _d Z _d ] (2)

Among them, X _d , Y _d and Z _d are the coordinate values of the 3D point cloud in the depth camera coordinate system, X _c , Y _c and Z _c are the coordinate values of the 3D point cloud in the first camera coordinate system, [R, T] _cd is a fixed transformation matrix between the depth camera and the first camera, and is a known quantity.

Specifically, when obtaining pose information, feature extraction is performed according to the depth map to obtain feature corner points, and descriptors corresponding to the feature corner points are determined. The descriptors of any two frames of the depth map are matched to obtain a match. Information, and obtain a transformation matrix according to the matching information; determine the three-dimensional point cloud in the second camera coordinate system according to the transformation matrix, and perform the transformation matrix according to the three-dimensional point cloud in the first camera coordinate system and the three-dimensional point cloud in the second camera coordinate system. Check; The pose information P is obtained by optimizing the transformed transformation matrix.

It should be noted that, in the embodiment of the present application, the three-dimensional point cloud in the depth camera coordinate system is constructed according to a depth map containing a depth value; the three-dimensional point cloud in the first camera coordinate system is based on the depth camera and the color camera. The fixed conversion relationship between them is obtained through calculation of the 3D point cloud in the depth camera coordinate system; the 3D point cloud in the second camera coordinate system is obtained through feature point matching according to the obtained color map, so theoretically, In the absence of a matching error, the 3D point cloud in the first camera coordinate system and the 3D point cloud in the second coordinate system should be the same.

It should be noted that, in this embodiment, feature corner points are obtained by performing feature extraction on the color map in the depth map, so the three-dimensional point cloud in the second camera coordinate system is based on the feature points in the color map. Extraction and matching. The 3D point cloud in the first camera coordinate system is obtained through the conversion of the 3D point cloud in the depth camera coordinate system. Although the acquisition methods are different, the two expressions have the same meaning, so for any feature corner point, its coordinates in the 3D point cloud of the first camera coordinate system should be the same as the 3D point cloud coordinates of the second camera coordinate system. the same. However, due to the existence of matching errors, the actual results may be different. Therefore, it is necessary to check the transformation matrix and optimize the transformed transformation matrix to make the obtained terminal pose information P more accurate. Of course, the specific way to obtain the pose information is not the focus of this application, so it will not be described in detail in the embodiments of this application.

Specifically, a three-dimensional point cloud in the world coordinate system is constructed based on the acquired pose information P and a three-dimensional point cloud in the first camera coordinate system to implement three-dimensional reconstruction. And use formula (3) to construct a three-dimensional point cloud in the world coordinate system. Formula (3) is expressed as follows:

[X _w Y _w Z _w ] = P [X _c Y _c Z _c ] (3)

Among them, X _w , Y _w and Z _w are the coordinate values of the three-dimensional point cloud in the world coordinate system, P is the acquired pose information, and X _c , Y _c and Z _c are the three-dimensional point cloud in the first camera coordinate system. Coordinate value.

It should be noted that when the coordinate system direction is determined, the standard image coordinate system is set to o ₁ -xy, then the relationship between the camera coordinate system and the pixel coordinate system is shown in Figure 2, and the relationship between the camera coordinate system and the world coordinate is as follows Shown in Figure 3.

Among them, as shown in FIG. 2, the rectangular coordinate system o-uv in pixels, which is established with the upper left corner of the depth image as the origin, is called a pixel coordinate system. The abscissa u and ordinate v of a pixel are the number of columns and the number of rows in its image array, respectively. The origin o _{1 of the} image coordinate system o ₁ -xy is defined as the intersection of the camera optical axis and the depth image plane, and the x-axis is parallel to the u-axis, and the y-axis is parallel to the v-axis. The camera coordinate system O _c -X _c Y _c Z _c uses the camera optical center O _c as the origin, and the X _c axis and Y _c axis are parallel to the x and y axes of the image coordinate system, respectively, and the Z _c axis is the optical axis of the camera , Perpendicular to the image plane and intersect at o ₁ point.

Among them, as shown in FIG. 3, the origin O _{w of the} world coordinate system O _w -X _w Y _w Z _w coincides with the origin O _{c of the} camera coordinate system, both of which are the camera optical centers, and the horizontal right is selected as the positive direction of the X _w axis. , The vertical downward direction is the positive direction of the Y _w axis, the vertical X _w Y _w plane and pointing directly forward is the positive direction of the Z _w axis, and a world coordinate system is established.

Specifically, when the road surface detection is performed according to the three-dimensional point cloud in the world coordinate system to determine the first path obstacle information, the road surface information is obtained based on the three-dimensional point cloud in the world coordinate system to obtain road surface information, and according to the acquired ground information Determine the first path obstacle information.

In a specific implementation, pavement detection is performed according to a three-dimensional point cloud in the world coordinate system, and a ground area and a non-ground area in the three-dimensional point cloud are determined, and the obtained result is used as ground information of the road. Obstacle detection is performed on the identified ground area to determine the path conditions and obstacle conditions on the ground area, and the three-dimensional point cloud in the world coordinate system is calibrated with [x, y, z], where x, y, and z represent The coordinates of any point in the three-dimensional point cloud in the world coordinate system, L represents the attribute of the point, L = 0 represents the path, L = 1 represents the obstacle, and the result of the detected path condition and obstacle condition is taken as the first Access obstacle information.

In step 103, image learning is performed according to the depth map to determine the second path obstacle information.

Specifically, an initial candidate region is generated according to the color map, and the initial candidate region is divided to obtain at least two first candidate regions; feature extraction is performed on each first candidate region, and a category of each first candidate region is determined, where The category includes obstacles and road surfaces; the second path obstacle information is determined according to the category of each first candidate area and the position of each first candidate area in the initial candidate area.

In a specific implementation, when initial candidate regions are generated through image learning on a color image, each initial candidate region is a rectangular frame and is represented by ROI = [a, b, w, h], a, b represents a rectangular frame initial candidate region The position coordinates of the left vertex in the color image, w, h represent the width and height of the rectangular frame of the initial candidate region. The initial candidate region ROI that has determined the position and size is evenly divided into k × k first candidate regions, and each box is represented by box = [a + i × k, b + j × k, w / k, h / k] The first candidate regions, i and j represent the serial numbers of each first candidate region in the horizontal direction and the vertical direction, respectively. Feature extraction is performed on each determined first candidate region box through convolution calculation to determine the category of each first candidate region, where the category is represented by the letter class. If the class class = 1 of the first candidate region is determined through feature extraction, it indicates that the first candidate region is an obstacle. If the class class = 0 of the first candidate region is determined through feature extraction, it indicates the first candidate region. For access. According to the category of each first candidate region and the position of each first candidate region in the initial candidate region, and the position of the initial candidate region in the color image, determine the information of obstacles and pathways in the color image, and the corresponding color Each pixel in the image corresponds to information about an obstacle and the road surface. According to the relationship between the pixel coordinate system, the camera coordinate system, and the world coordinate system, the corresponding point in the world coordinate system corresponding to the pixel in the color image is determined. Thereby, the second path and obstacle information are determined.

In step 104, the first-path obstacle information and the second-path obstacle information are combined to obtain third-path obstacle information.

Specifically, the first-path obstacle information and the second-path obstacle information are combined, that is, the points determined as obstacles and paths in the first-path obstacle information and the second-path obstacle information are retained, and the first Only one of the obstacle information of the first passage or the obstacle information of the second passage is determined to be the obstacle and the passage point is also retained, thereby obtaining the obstacle information of the third passage. Therefore, the third-path obstacle information includes both the entire contents of the first-path obstacle information and the second-path obstacle information. Combining the two aspects of localization and image learning to obtain path information and obstacle information makes the obtained results more comprehensive and avoids the lack of path information and obstacle information.

In step 105, route navigation is performed based on the third-path obstacle information.

Compared with the prior art, the path navigation method provided in this embodiment determines the first path obstacle information determined by road detection based on the acquired depth map, and combines the second path obstacle information determined by image learning based on the depth map to determine The third path obstacle information is more matched with the actual road conditions, and navigation is performed according to the third path obstacle information, thereby improving the accuracy of path navigation.

The second embodiment of the present application relates to a route navigation method. This embodiment is further improved based on the first embodiment. The specific improvement is as follows: Detailed description. The flow of the route navigation method in this embodiment is shown in FIG. 4.

Specifically, in this embodiment, steps 201 to 208 are included, where steps 201 to 204 are substantially the same as steps 101 to 104 in the first embodiment, and are not repeated here. The following mainly describes the differences. For technical details that are not described in detail in this embodiment mode, refer to the path navigation method provided in the first embodiment, and details are not described herein again.

After step 201 to step 204, step 205 is performed.

In step 205, a two-dimensional grid map is obtained according to the three-dimensional point cloud in the world coordinate system.

Specifically, in this embodiment, a three-dimensional point cloud in a world coordinate system is projected onto a road surface to form a two-dimensional grid map, and path navigation is performed based on the two-dimensional grid map.

In step 206, the attributes of each grid in the two-dimensional grid map are determined according to the third path obstacle information.

Specifically, the attributes of each grid in the two-dimensional grid map are determined according to the obtained third-path obstacle information, where the attributes include obstacles and paths. And its attributes can be calibrated in a two-dimensional grid map with different colors. For example, use SG (p, q) = 1 to represent the grid of obstacles in the two-dimensional grid map and use white for calibration; use SG (p, q) = 0 to represent the pathways in the two-dimensional grid map. Grid and calibrate with black.

In step 207, the location of the destination and the current location are determined.

Specifically, in this embodiment, a current position may be determined by a Global Positioning System (GPS), and a position of a destination to be reached may be determined according to a received user instruction.

In step 208, the optimal path is determined according to the location of the destination, the current location, and the attributes of each grid.

In a specific implementation, if the location of the destination determined in the two-dimensional grid map is T (u ₁ , v ₁ ), and the determined current location is G (u ₀ , v ₀ ), then according to the destination's Calculate a straight line for the position and the current position as L ₁ , and determine the straight line equation of L ₁ as u = A ₁ v + B ₁ , where A ₁ = (u ₁ -u ₀ ) / (v ₁ -v ₀ ), B ₁ = U ₁ -A ₁ × v ₁ . Taking the current position G (u ₀ , v ₀ ) as the center of the circle, a straight line L _{2 is} given arbitrarily, and the straight line equation of L ₂ is u = A ₂ v + B ₂ . The angle between the straight line L ₁ and the straight line L ₂ is θ, and satisfies tan θ = | (A ₁ -A ₂ ) / (1 + A ₁ × A ₂ ) |, A ₂ can be obtained, and then B ₂ = u _0- A ₂ × v ₀ . Starting from the current position G (u ₀ , v ₀ ), iterates through the points on the straight line L ₂ and counts the number of grids with the attribute SG (p, q) = 0. When θ is satisfied and the attribute is SG (p , q) = 0 When the total number of grids is greater than the preset threshold, it is determined that θ is the optimal forward direction, and the moving step is the total number of grids with the attribute SG (p, q) = 0 multiplied by m, where m is the actual physical size corresponding to each grid, and the unit is meter.

A third embodiment of the present application relates to a route navigation device, and a specific structure thereof is shown in FIG. 5.

As shown in FIG. 5, the path navigation device includes an acquisition module 301, a first determination module 302, a second determination module 303, a merge module 304, and a navigation module 305.

The obtaining module 301 is configured to obtain a depth map.

The first determining module 302 is configured to construct a three-dimensional point cloud in the world coordinate system according to the depth map, and perform road surface detection according to the three-dimensional point cloud in the world coordinate system to determine the first path obstacle information.

The second determining module 303 is configured to perform image learning according to the depth map to determine the second path obstacle information.

The merging module 304 is configured to merge the first-path obstacle information and the second-path obstacle information to obtain the third-path obstacle information.

The navigation module 305 is configured to perform route navigation according to the third path obstacle information.

It is not difficult to find that this embodiment is a device example corresponding to the first embodiment, and this embodiment can be implemented in cooperation with the first embodiment. Relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, details are not repeated here. Accordingly, the related technical details mentioned in this embodiment can also be applied in the first embodiment.

The fourth embodiment of the present application relates to a route navigation device. This embodiment is substantially the same as the third embodiment. The specific structure is shown in FIG. 6. Among them, the main improvement is that the fourth embodiment specifically describes the structure of the navigation module in the third embodiment.

The navigation module 305 includes an acquisition sub-module 3051, an attribute determination sub-module 3052, a position determination sub-module 3053, and a path determination sub-module 3054.

An obtaining sub-module 3051 is configured to obtain a two-dimensional grid map according to a three-dimensional point cloud in the world coordinate system.

The attribute determining sub-module 3052 is configured to determine an attribute of each grid in the two-dimensional grid map according to the obstacle information of the third path.

The position determining submodule 3053 is configured to determine the position of the destination and the current position.

The path determination sub-module 3054 is configured to determine an optimal path according to the destination position, the current position, and the attributes of each grid.

It is not difficult to find that this embodiment is a device example corresponding to the second embodiment, and this embodiment can be implemented in cooperation with the second embodiment. Relevant technical details mentioned in the second embodiment are still valid in this embodiment, and in order to reduce repetition, details are not repeated here. Accordingly, the related technical details mentioned in this embodiment can also be applied in the second embodiment.

The device embodiments described above are only schematic and do not limit the scope of protection of this application. In practical applications, those skilled in the art may select some or all of the modules to implement this embodiment according to actual needs. The purpose of the plan is not limited here.

A fifth embodiment of the present application relates to an electronic device, and a specific structure thereof is shown in FIG. 7. Includes at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501. The memory 502 stores instructions executable by the at least one processor 501, and the instructions are executed by the at least one processor 501, so that the at least one processor 501 can execute a route navigation method.

In this embodiment, the processor 501 uses a central processing unit (CPU) as an example, and the memory 502 uses a readable and writable memory (Random Access Memory, RAM) as an example. The processor 501 and the memory 502 may be connected through a bus or in other manners. In FIG. 7, the connection through the bus is taken as an example. The memory 502 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. For example, the program for implementing the method for determining environmental information in the embodiment of the present application is Stored in the memory 502. The processor 501 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory 502, that is, the above path navigation method is implemented.

The memory 502 may include a storage program area and a storage data area, where the storage program area may store an operating system and an application program required for at least one function; the storage data area may store a list of options and the like. In addition, the memory may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely set relative to the processor 501, and these remote memories may be connected to an external device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

One or more program modules are stored in the memory 502, and when executed by one or more processors 501, execute the path navigation method in any of the above method embodiments.

The above products can execute the method provided in the embodiment of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details not described in this embodiment, refer to the method provided in the embodiment of the present application.

An eighth embodiment of the present application relates to a computer-readable storage medium. The computer-readable storage medium stores a computer program that can implement a path navigation method involved in any method embodiment of the present application when the computer program is executed by a processor. .

Those skilled in the art can understand that all or part of the steps in the method of the above embodiments can be completed by a program instructing related hardware. The program is stored in a storage medium and includes several instructions for making a device (which can be (Single chip microcomputer, chip, etc.) or a processor (processor) executes all or part of the steps of the method described in each embodiment of the present application. The foregoing storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes .

Those of ordinary skill in the art can understand that the foregoing embodiments are specific embodiments for implementing the present application, and in practical applications, various changes can be made in form and details without departing from the spirit and range.

Claims

A route navigation method includes:

Get depth map;

Constructing a three-dimensional point cloud in a world coordinate system according to the depth map, and performing road surface detection according to the three-dimensional point cloud in the world coordinate system to determine first-path obstacle information;

Performing image learning according to the depth map to determine obstacle information of the second path;

Combining the first-path obstacle information and the second-path obstacle information to obtain third-path obstacle information;

Path navigation is performed according to the third path obstacle information.
The path navigation method according to claim 1, wherein the depth map comprises: a color map and a depth value corresponding to the color map.
The path navigation method according to claim 2, wherein the three-dimensional point cloud in the world coordinate system is constructed according to the depth map, and the first path obstacle is determined according to the road surface detection based on the three-dimensional point cloud in the world coordinate system. Physical information, including:

Constructing a three-dimensional point cloud in the first camera coordinate system according to the depth map;

Obtain pose information;

Constructing a three-dimensional point cloud in a world coordinate system according to the three-dimensional point cloud in the first camera coordinate system and the pose information;

Performing road surface detection according to the three-dimensional point cloud in the world coordinate system to determine the first path obstacle information.
The path navigation method according to claim 3, wherein the acquiring pose information specifically comprises:

Performing feature extraction according to the depth map to obtain feature corner points, and determining a descriptor corresponding to the feature corner points;

Matching the descriptors of any two frames of the image in the depth map to obtain matching information;

Obtaining a conversion matrix according to the matching information;

Determining a three-dimensional point cloud in a second camera coordinate system according to the transformation matrix;

Verifying the transformation matrix according to the three-dimensional point cloud in the first camera coordinate system and the three-dimensional point cloud in the second camera coordinate system;

The pose information is obtained by optimizing the verified transformation matrix.
The path navigation method according to any one of claims 3 to 4, wherein the determining the first path obstacle information by performing road surface detection based on the three-dimensional point cloud in the world coordinate system specifically includes:

Performing ground detection according to the three-dimensional point cloud in the world coordinate system to obtain ground information of a road;

Determining the first path obstacle information according to the acquired ground information.
The path navigation method according to claim 5, wherein the performing image learning to determine the second path obstacle information according to the depth map specifically comprises:

Generating an initial candidate region according to the color map;

Dividing the initial candidate region to obtain at least two first candidate regions;

Performing feature extraction on each of the first candidate areas, and determining a category of each of the first candidate areas, where the categories include obstacles and road surfaces;

Determining the second path obstacle information according to the category of each of the first candidate regions and the position of each of the first candidate regions in the initial candidate region.
The route navigation method according to claim 6, wherein the performing route navigation according to the third path obstacle information specifically comprises:

Obtaining a two-dimensional grid map according to a three-dimensional point cloud in the world coordinate system;

Determining the attributes of each grid in the two-dimensional grid map according to the third path obstacle information, wherein the attributes include obstacles and paths;

Determine the location and current location of the destination;

An optimal path is determined according to the location of the destination, the current location, and the attributes of each grid.
A path navigation device includes:

An acquisition module for acquiring a depth map;

A first determining module, configured to construct a three-dimensional point cloud in a world coordinate system according to the depth map, and perform road surface detection according to the three-dimensional point cloud in the world coordinate system to determine first obstacle information;

A second determining module, configured to perform image learning and determine second-path obstacle information according to the depth map;

A merging module, configured to combine the first-path obstacle information and the second-path obstacle information to obtain third-path obstacle information;

A navigation module is configured to perform route navigation according to the third path obstacle information.
An electronic device includes:

At least one processor; and

A memory connected in communication with the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of claims 1 to 7. Path navigation method.
A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the path navigation method according to any one of claims 1 to 7 is implemented.