WO2019148311A1

WO2019148311A1 - Information processing method and system, cloud processing device and computer program product

Info

Publication number: WO2019148311A1
Application number: PCT/CN2018/074514
Authority: WO
Inventors: 高军强; 廉士国; 林义闽
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2018-01-30
Filing date: 2018-01-30
Publication date: 2019-08-08
Also published as: CN108401462A

Abstract

Provided in embodiment of the present application are an information processing method and system, a cloud processing device and a computer program product, relating to the technical field of information processing. The calculating pressure of a terminal is reduced to a certain extent and mapping efficiency is improved. The information processing method provided in the embodiment of the present application comprises: acquiring by the cloud processing device an original three-dimensional map data generated by a terminal; extracting pose information in the original three-dimensional map data and performing gridding processing on the original three-dimensional map data to generate a first high resolution map corresponding to the pose information, the resolution of the first high resolution map being higher than that of the original three-dimensional map data; and sending the first high resolution map to the terminal.

Description

Information processing method and system, cloud processing device and computer program product

Technical field

The present application relates to the field of information processing technologies, and in particular, to an information processing method and system, a cloud processing device, and a computer program product.

Background technique

With the rapid development of Internet of Things technology, new data computing modes such as pervasive computing, holographic computing, and cloud computing are gradually entering people's daily lives, which can be applied to many fields. Among them, computer vision can be a representative field. Computer vision is a science that studies how to make a machine "look". Further, it refers to the use of equipment instead of the human eye to identify, track, and measure the machine vision, and further image processing. It is more suitable for human eyes to observe or transmit images detected by the instrument.

Among them, 3D map reconstruction is an important manifestation of computer vision. 3D map reconstruction is a mathematical model suitable for computer representation and understanding. It is the basis for computer processing, operation and analysis of 3D space environment. The key technology of creating virtual reality in the objective world is established in computers.

In the prior art, in the process of three-dimensional map reconstruction, a large amount of data needs to be processed. Since the terminal is limited by the resources of the computing unit, processing large-scale three-dimensional data is slow and has low precision.

Summary of the invention

The embodiment of the present application provides an information processing method and system, a cloud processing device, and a computer program product, and the cloud processing device is used to complete a partial three-dimensional map reconstruction process, which can reduce the calculation pressure of the terminal and improve the efficiency of the mapping.

In a first aspect, an embodiment of the present application provides an information processing method, including:

The cloud processing device acquires original three-dimensional map data generated by the terminal;

Extracting pose information in the original three-dimensional data, performing meshing processing on the original three-dimensional map data, and generating a first high-resolution map corresponding to the pose information;

Sending the first high resolution map to the terminal.

In a second aspect, the embodiment of the present application further provides an information processing system, including:

An obtaining unit, configured to acquire original three-dimensional map data generated by the terminal;

a processing unit, configured to extract pose information in the original three-dimensional data, perform meshing processing on the original three-dimensional map data, and generate a first high-resolution map corresponding to the pose information;

And a sending unit, configured to send the first high resolution map to the terminal.

In a third aspect, the embodiment of the present application further provides a cloud processing device, where the device includes a processor and a memory; the memory is configured to store an instruction, when the instruction is executed by the processor, causing the device to perform, for example, The method of any of the first aspects.

In a fourth aspect, the embodiment of the present application further provides a computer program product, which can be directly loaded into an internal memory of a computer and includes software code. After the computer program is loaded and executed by a computer, the first aspect can be implemented. One such method.

The information processing method and system, the cloud processing device, and the computer program product provided by the embodiment of the present application, after acquiring the original three-dimensional map data generated by the terminal by the cloud processing device, extracting the pose information in the original three-dimensional map data, and according to the pose The information is used to mesh the original three-dimensional map data, generate a first high-resolution map corresponding to the pose information, and then send the first high-resolution map to the terminal, since the first high-resolution map is processed in the cloud The computing pressure generated by the device reduces the computing pressure caused by the processing of a large amount of data by the terminal, and the calculation of the cloud processing device has the characteristics of high precision and fast processing speed, and solves the prior art in some cases, because the terminal is subjected to the computing unit. Resource limitations, slower, and less accurate when dealing with large-scale 3D data.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative labor for those skilled in the art.

FIG. 1 is a flowchart of an embodiment of an information processing method according to an embodiment of the present application;

FIG. 2 is another flowchart of an embodiment of an information processing method according to an embodiment of the present disclosure;

FIG. 3 is another flowchart of an embodiment of an information processing method according to an embodiment of the present disclosure;

FIG. 4 is another flowchart of an embodiment of an information processing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure;

FIG. 6 is another schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure;

FIG. 7 is another schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of an embodiment of a cloud processing device according to an embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The terms used in the embodiments of the present application are for the purpose of describing particular embodiments only, and are not intended to limit the application. "an,""an,"""""""

It should be understood that the term "and/or" as used herein is merely an association describing the associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, while A and B, there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting." Similarly, depending on the context, the phrase "if determined" or "if detected (conditions or events stated)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event) "Time" or "in response to a test (condition or event stated)".

At present, real-time 3D mapping has become a hot research field. The purpose of real-time 3D mapping is to strengthen the perception and understanding of computers and other devices to the surrounding environment. It is the core issue of robot navigation, augmented reality, virtual reality and other scenarios. Due to the resource limitation of the computing unit, the problem of using the real-time three-dimensional mapping of the computing unit in the device is that the resolution is low, the precision is not high, and the accuracy is not high. Therefore, in order to solve the foregoing problem, the embodiment of the present application provides an information processing method. After the terminal performs three-dimensional reconstruction of the collected environment information to generate a three-dimensional map, the cloud processing device processes the three-dimensional map generated by the terminal in real time, and generates the terminal. The low-resolution map is converted into a high-resolution map. Specifically, FIG. 1 is a flowchart of an embodiment of the information processing method provided by the embodiment of the present application. As shown in FIG. 1 , the information processing method provided by the embodiment of the present application, Specifically, the method may include the following steps:

101. The cloud processing device acquires original three-dimensional map data generated by the terminal.

In the embodiment of the present application, the terminal collects environment information through the collection device, and obtains corresponding device positioning and environment three-dimensional data. The collection device includes a positioning acquisition device and a three-dimensional image acquisition device, and may be a stereo camera, a depth camera or the like, or a laser radar sensor. Then, the obtained positioning data is calculated by using a slam (simultaneous localization and mapping) positioning algorithm on the terminal, and the pose information of the collecting device is obtained, and then, according to the pose information and the corresponding environment three-dimensional data. The reconstruction is performed to obtain three-dimensional reconstruction data, that is, original three-dimensional map data. It should be noted that, depending on the positioning device used, in the embodiment of the present application, the device pose information is obtained by using the visual slam or the laser slam algorithm, and the reconstructed three-dimensional data preferably uses the point cloud data before the meshing. It can be understood that, in the embodiment of the present invention, the original three-dimensional map data includes pose information and point cloud data.

Then, the terminal sends the real-time generated three-dimensional map data to the cloud processing device, so that the cloud processing device can receive the original three-dimensional map data sent by the terminal.

In the embodiment of the present application, the terminal may be controlled by the user, and the purpose thereof is to construct a three-dimensional map of a complex environment, for example, an indoor space map of a shopping mall or the like.

102. Extract the pose information in the original three-dimensional data, perform mesh processing on the point cloud data in the original three-dimensional map data, and generate a first high-resolution map corresponding to the pose information.

In a specific implementation process, the correct topological connection relationship of each point cloud data is reconstructed by interpolation or approximation of the surface on which the data points are located, thereby obtaining a triangular mesh model similar to the surface topology of the original object. Then, according to the preset resolution, the size of the grid is determined to obtain a corresponding first high resolution map.

It should be noted that, in the embodiment of the present application, after the terminal sends the original three-dimensional data to the cloud processing device or the terminal sends the original three-dimensional data to the cloud processing device, the terminal also generates a low-resolution three-dimensional map, and in the cloud. The resolution and accuracy of the first high resolution map generated by the processing device is higher than the resolution of the low resolution three dimensional map.

103. Send the first high resolution map to the terminal.

In the embodiment of the present application, the cloud processing device may send the first high resolution map to the terminal immediately after generating the corresponding first high resolution map, or may set the first high resolution map according to a preset time interval. Send to the terminal.

In the implementation of the application, the terminal may further be provided with a display device, so that the user can view the first high-resolution map returned by the cloud processing device, and then control the movement of the terminal according to the corresponding content, and assist the user to determine whether the device needs to be re-acquired. Data to get more accurate data.

The information processing method provided by the embodiment of the present application obtains the original three-dimensional map data generated by the terminal by the cloud processing device, extracts the pose information in the original three-dimensional map data, and uses the pose information to point the point cloud in the original three-dimensional map. The data is meshed to generate a first high resolution map corresponding to the pose information, and then the first high resolution map is sent to the terminal, since the first high resolution map is generated in the cloud processing device, To reduce the computational pressure brought by the terminal to process a large amount of data, and the calculation of the cloud processing device has the characteristics of high precision and fast processing speed, and to some extent, the prior art is solved, because the terminal is limited by the resource of the computing unit, and the processing is large. Scale 3D data will be slower and less accurate.

On the basis of the foregoing content, further, in the embodiment of the present application, the terminal may move the location by itself, or the user may control the mobile terminal to move the location, so that the terminal may collect different environmental information during the mobile process, and then the terminal may collect according to the environment. The different environmental information is reconstructed to generate at least one original three-dimensional map data, and correspondingly, the cloud processing device receives at least one original three-dimensional map data. Because the posture information changes during the movement of the terminal, the cloud processing device processes the first high-resolution map according to the change of the posture information. Specifically, FIG. 2 is an information processing method provided by the embodiment of the present application. Another flowchart of the embodiment, as shown in FIG. 2, the information processing method provided by the embodiment of the present application may further include the following steps:

104. According to the change of the pose information, the first high-resolution map corresponding to the different pose information is spliced to generate a second high-resolution map.

In the embodiment of the present application, after extracting the pose information, the cloud processing device processes and labels the pose information to determine the location of the terminal, and then the cloud processing device splices the first high-resolution map at different locations. A continuous map is formed, that is, a second high resolution map is generated.

In the embodiment of the present application, after the cloud processing device generates a plurality of first high-resolution maps, the pose information in each first high-resolution map is extracted, and then the pose in the pose information is used as an initial position. The ICP iterative algorithm is used to iteratively optimize the point cloud data in the first high resolution map to generate a second high resolution map corresponding to the pose information. It is worth mentioning that the pose information in the original three-dimensional map is optimized as the initial pose of the cloud ICP algorithm, which can greatly save the time of cloud 3D map construction and improve the construction efficiency.

105. Send the second high resolution map to the terminal.

Further, since the pose information of the terminal may have a certain error, the second high-resolution map generated according to the pose information of the terminal may also have a certain error. Therefore, in order to improve the accuracy of the high-resolution map, Before the cloud processing device sends the second high-resolution map to the terminal, the following steps may be further included. Specifically, FIG. 3 is another flowchart of the information processing method provided by the embodiment of the present application, as shown in FIG. The information processing method provided by the embodiment of the present application may further include the following steps before the step 105:

106. Optimize the second high resolution map.

In the embodiment of the present application, in the process of splicing the first high-resolution map, there will be a certain error, and the accumulation of errors will generate a large cumulative error. Therefore, the constraint relationship can be established and reduced by nonlinear optimization. The effect of the cumulative error results in a more accurate three-dimensional map. For example, in a specific implementation process, loopback detection can be used to reduce the cumulative error in the 3D mapping process. For example, in the process of splicing the first high resolution map to form the second high resolution map, an overlapping portion is generated, and the overlapping portion of the second high resolution map is removed.

Correspondingly, step 105 is performed as: “send the processed second high resolution map to the terminal”.

In the embodiment of the present application, when a large-scale three-dimensional map needs to be established, at least two terminals may be required to collect environmental information, and then the data of each terminal is separately processed, spliced, etc., to obtain a large-scale three-dimensional map. Specifically, FIG. 4 is another flowchart of an embodiment of an information processing method according to an embodiment of the present disclosure. As shown in FIG. 4, the information processing method provided by the embodiment of the present application may specifically include the following steps:

201. Acquire original three-dimensional map data generated by each terminal.

In the embodiment of the present application, each terminal independently collects environmental information, and then performs positioning reconstruction separately to obtain respective original three-dimensional map data, and then respectively sends corresponding original three-dimensional map data to the cloud processing device.

In addition, in the embodiment of the present application, when a plurality of terminals jointly complete a large-scale three-dimensional map reconstruction, the pose relationship between each terminal is pre-calibrated, and the terminal calculates the respective poses in real time during the movement. When the cloud processing device extracts the pose information, the relative position between the terminals can be calculated. In a specific implementation process, a global coordinate system is established. Before collecting environmental information, the relationship between the body coordinate system of each terminal and the global coordinate system is first calibrated, and then the position of each terminal in the global coordinate system is calculated in real time. In the process of reconstructing the original three-dimensional map data, the terminal can reconstruct according to the coordinates of the global coordinate system.

In the embodiment of the present application, the cloud processing device separately receives the original three-dimensional map data sent by each terminal.

202. Extract the pose information in each original three-dimensional map data, respectively process the point cloud data in the original three-dimensional map data, and respectively generate a first high-resolution map corresponding to the pose information.

In the embodiment of the present application, since the processing of the original three-dimensional map data to generate the first high-resolution map requires a large amount of calculation, in order to speed up the calculation, a GPU (Graphics Processing Unit) may be used. Calculation. It is also possible to create different threads for parallel computing.

When the cloud processing device receives the plurality of original three-dimensional map data, respectively extracts the pose information in each original three-dimensional map data, and then separately processes the point cloud data in each original three-dimensional map data to generate the first High resolution map.

203. Splicing the first high-resolution map corresponding to different pose information according to a relative positional relationship between different pose information to generate a third high-resolution map.

As mentioned in the foregoing, the global coordinate system is pre-established, and each time a first high resolution map is generated, the position of the first high resolution map in the global coordinate system can be obtained. Therefore, after the plurality of first high-resolution maps are generated, the plurality of first high-resolution maps may be spliced according to the global coordinate system to obtain a third high-resolution map.

Preferably, when a plurality of first high-resolution maps overlap, a constraint relationship may be established for nonlinear optimization, which reduces cumulative errors in each device positioning process, improves three-dimensional mapping accuracy, and removes overlapping regions, for example, The loopback detection algorithm is used to optimize the third high resolution map to improve the accuracy of the three-dimensional map and remove the overlapping regions.

204. Send the third high resolution map to the terminal.

In the embodiment of the present application, after receiving the third high-resolution map, the terminal may repair the low-resolution three-dimensional map generated by itself according to the third high-resolution map, and the low-resolution three-dimensional resolution with lower resolution. The map is corrected to a third high resolution map with a higher resolution. The terminal can also delete the low-resolution three-dimensional map generated by itself and directly use the received third high-resolution map.

The embodiment of the present application further provides an information processing system, and FIG. 5 is a schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 5, the system in this embodiment may be used. The acquisition unit 11 includes a retrieval unit 11, a processing unit 12, and a transmission unit 13.

The obtaining unit 11 is configured to acquire original three-dimensional map data generated by the terminal.

The processing unit 12 is configured to extract pose posture information in the original three-dimensional data, perform mesh processing on the original three-dimensional map data, and generate a first high-resolution map corresponding to the pose information;

The sending unit 13 is configured to send the first high resolution map to the terminal.

FIG. 6 is another schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 6, the system in this embodiment may further include: a splicing unit 14 on the basis of the foregoing content.

The splicing unit 14 is configured to splicing the first high-resolution map corresponding to different pose information according to the change of the pose information to generate a second high-resolution map.

The sending unit 13 is further configured to send the second high resolution map to the terminal.

FIG. 7 is another schematic structural diagram of an embodiment of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 7, the system in this embodiment may further include: an optimization unit 15 on the basis of the foregoing content.

The optimization unit 15 is configured to perform optimization processing on the second high resolution map.

The sending unit 13 is further configured to:

The processed second high resolution map is sent to the terminal.

In a specific implementation process, there are at least two terminals;

The obtaining unit 11 is specifically configured to:

Obtaining original three-dimensional map data generated by each terminal;

The extracting unit 12 is specifically configured to:

The pose information in each original three-dimensional map data is extracted, and the original three-dimensional map data is separately processed to generate a first high-resolution map corresponding to the pose information.

The splicing unit 14 is further configured to:

Separating the first high-resolution maps corresponding to different pose information according to the relative positional relationship between different pose information to generate a third high-resolution map;

The sending unit 13 is further configured to:

The third high resolution map is sent to the terminal.

The system of the present embodiment can be used to implement the technical solution of the method embodiment shown in FIG. 1 to FIG. 4 , and the implementation principle and technical effects are similar, and details are not described herein again.

The embodiment of the present application further provides a cloud processing device, and FIG. 8 is a schematic structural diagram of an embodiment of a cloud processing device according to an embodiment of the present disclosure. The cloud processing device includes a processor 21 and a memory 22; the memory 22 is for storing instructions that, when executed by the processor 21, cause the device to perform any of the methods described above.

The cloud processing device provided by the embodiment of the present application may be used to implement the technical solution of the method embodiment shown in FIG. 1 to FIG. 4 , and the implementation principle and technical effects thereof are similar, and details are not described herein again.

In order to implement the method flow of the foregoing, the embodiment of the present application further provides a computer program product, which can be directly loaded into an internal memory of a computer and contains software code, and the computer program can be implemented by being loaded and executed by a computer. Any method.

The computer program product provided by the embodiment of the present application can be used to implement the technical solution of the method embodiment shown in FIG. 1 to FIG. 4 , and the implementation principle and the technical effect are similar, and details are not described herein again.

One of ordinary skill in the art will appreciate that all or part of the steps to implement the various method embodiments described above may be accomplished by hardware associated with the program instructions. The aforementioned program can be stored in a computer readable storage medium. The program, when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

The device embodiments described above are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located in one place. Or it can be distributed to at least two network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Finally, it should be noted that the above embodiments are only for explaining the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present application. range.

Claims

An information processing method, comprising:

The cloud processing device acquires original three-dimensional map data generated by the terminal;

Extracting pose information in the original three-dimensional data, performing meshing processing on the original three-dimensional map data, and generating a first high-resolution map corresponding to the pose information; and the first high-resolution map Send to the terminal.
The method of claim 1 further comprising:

And splicing the first high-resolution maps corresponding to the different pose information according to the change of the pose information to generate a second high-resolution map;

Sending the second high resolution map to the terminal.
The method of claim 2, wherein before the sending the second high resolution map to the terminal, the method further comprises:

Optimizing the second high resolution map;

Sending the second high resolution map to the terminal includes:

The processed second high resolution map is sent to the terminal.
The method of claim 1 wherein said terminal has at least two;

And acquiring the original three-dimensional map data generated by the terminal, including:

Obtaining original three-dimensional map data generated by each of the terminals;

Extracting the pose information in the original three-dimensional data, and processing the original three-dimensional map data to generate a first high-resolution map corresponding to the pose information, including:

Extracting the pose information in each of the original three-dimensional map data, respectively processing the original three-dimensional map data, and respectively generating a first high-resolution map corresponding to the pose information.
The method of claim 4, wherein the method further comprises:

Separating the first high-resolution maps corresponding to the different pose information according to different relative positional relationships between the pose information to generate a third high-resolution map;

Sending the first high resolution map to the terminal includes:

Sending the third high resolution map to the terminal.
An information processing system, comprising:

An obtaining unit, configured to acquire original three-dimensional map data generated by the terminal;

a processing unit, configured to extract pose information in the original three-dimensional data, perform meshing processing on the original three-dimensional map data, and generate a first high-resolution map corresponding to the pose information; And transmitting the first high resolution map to the terminal.
The system of claim 6 wherein the system further comprises:

a splicing unit, configured to splicing the first high-resolution maps corresponding to the different pose information according to the change of the pose information to generate a second high-resolution map;

The sending unit is further configured to send the second high resolution map to the terminal.
The system of claim 7 wherein said system further comprises:

An optimization unit, configured to optimize the second high resolution map;

The sending unit is further configured to:

The processed second high resolution map is sent to the terminal.
The system of claim 6 wherein said terminal has at least two;

The obtaining unit is specifically configured to:

Obtaining original three-dimensional map data generated by each of the terminals;

The extracting unit is specifically configured to:

Extracting the pose information in each of the original three-dimensional map data, respectively processing the original three-dimensional map data, and respectively generating a first high-resolution map corresponding to the pose information.
The system according to claim 9, wherein the splicing unit is further configured to:

Separating the first high-resolution maps corresponding to the different pose information according to different relative positional relationships between the pose information to generate a third high-resolution map;

The sending unit is further configured to:

Sending the third high resolution map to the terminal.
A cloud processing device, characterized in that the device comprises a processor and a memory; the memory is for storing instructions, when the instructions are executed by the processor, causing the device to perform as claimed in claims 1 to 5 Any of the methods described.
A computer program product, which can be directly loaded into an internal memory of a computer and containing software code, which can be implemented by any one of claims 1 to 5 after being loaded and executed by a computer Methods.