WO2021189194A1

WO2021189194A1 - Three-dimensional environment modeling method and device, computer storage medium, and industrial robot operating platform

Info

Publication number: WO2021189194A1
Application number: PCT/CN2020/080684
Authority: WO
Inventors: 丁万; 察普夫·马克·帕特里克
Original assignee: 罗伯特博世有限公司; 丁万
Priority date: 2020-03-23
Filing date: 2020-03-23
Publication date: 2021-09-30
Also published as: DE112020006265T5; CN115244581A

Abstract

The present invention provides a three-dimensional environment modeling method. The method comprises: scanning an ambient environment to obtain a point cloud flow; detecting one or more augmented reality codes in the ambient environment; and carrying out improvement and optimization operations on the point cloud flow at least on the basis of information carried by the one or more augmented reality codes and positions of the one or more augmented reality codes, so as to perform modeling on the ambient environment. The present invention further relates to a three-dimensional environment modeling device, a computer storage medium, and an industrial robot operating platform.

Description

Three-dimensional environment modeling method and equipment, computer storage medium and industrial robot operating platform

【Technical Field】

The invention relates to a three-dimensional environment modeling method and equipment, a computer storage medium and an industrial robot operating platform.

【Background technique】

In some specific industrial application scenarios, try to use a camera (such as an RGB-D camera) to scan the environment and use the scanned and recorded data to reconstruct the environment. However, in the subsequent environmental reconstruction, a lot of manual work is usually involved, especially during post-processing, the user must specify the boundary of the working area in the x, y, or z plane, and delete the scenes that existed at the time of recording but did not belong to the scene. Temporary 3D objects, or fill objects that are only partially scanned.

In addition, the depth sensor of the RGB-D camera may encounter problems when capturing reflective surfaces such as glass. For example, the part of the reconstructed three-dimensional model that was originally a solid surface may be missing or there may be noise.

Therefore, an improved three-dimensional environment modeling solution is desired.

[Summary of the invention]

An aspect of the present invention provides a three-dimensional environment modeling method, the method includes: scanning the surrounding environment to obtain a point cloud stream; detecting one or more augmented reality codes in the surrounding environment; and at least based on The information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes are used to operate the point cloud stream in order to model the surrounding environment (so as to achieve accelerated modeling And automatically optimize and perfect the point cloud stream operation).

Preferably, in the above-mentioned three-dimensional environment modeling method, scanning the surrounding environment includes: using an RGB-D camera to scan the surrounding environment from multiple angles.

Preferably, the aforementioned three-dimensional environment modeling method may further include: recording the point cloud flow.

Preferably, in the above-mentioned three-dimensional environment modeling method, detecting one or more augmented reality codes in the surrounding environment includes: when scanning the surrounding environment, using an augmented reality tracking library to detect one of the surrounding environments Or a plurality of augmented reality codes and their degrees of freedom poses relative to the RGB-D camera; and recording the degrees of freedom poses and registering them in a point cloud coordinate system.

Preferably, in the above-mentioned three-dimensional environment modeling method, the one or more augmented reality codes are placed on the surface of the target object.

Preferably, in the above-mentioned three-dimensional environment modeling method, operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes includes: According to the information carried by the one or more augmented reality codes, a geometric structure is constructed at the position of the one or more augmented reality codes, wherein the orientation of the geometric structure with respect to the one or more augmented reality codes And the content of the geometric structure is encoded in the information.

Preferably, in the above-mentioned three-dimensional environment modeling method, the geometric structure is a polygonal boundary/wall, a rectangular parallelepiped, a cylinder, or the like.

Preferably, in the above-mentioned three-dimensional environment modeling method, operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes includes: According to the information carried by the one or more augmented reality codes, remove obstacles or boundaries at or near the location of the one or more augmented reality codes in the point cloud stream, and the area of the point to be removed The range and orientation of is encoded in the information.

Preferably, in the above-mentioned three-dimensional environment modeling method, operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes includes: Perform region operations on the point cloud stream, where the region operations include region segmentation, filtering and/or clustering of the target object, and the parameters of the operation are defined in the one or more augmented reality codes carried Information.

Another aspect of the present invention provides a three-dimensional environment modeling method, the method includes: scanning the surrounding environment of the industrial robot to obtain a point cloud flow; detecting one or more enhancements in the surrounding environment Reality code; operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes; and converting the point cloud stream after the operation It is processed into a surface mesh for modeling of the surrounding environment (so as to achieve accelerated modeling and automatic optimization and perfect point cloud flow operations).

Another solution of the present invention provides a three-dimensional environment modeling device, the device includes: a scanning device for scanning the surrounding environment, so as to obtain a point cloud flow; detection device for detecting the surrounding environment One or more augmented reality codes; and an execution device for operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes To model the surrounding environment.

Preferably, in the above-mentioned three-dimensional environment modeling device, the scanning device is configured to scan the surrounding environment from multiple angles using an RGB-D camera.

Preferably, the above-mentioned three-dimensional environment modeling equipment may further include: a recording device for recording the point cloud flow.

Preferably, in the above-mentioned three-dimensional environment modeling equipment, the detection device includes: a first detection unit configured to use an augmented reality tracking library to detect one or more of the surrounding environments when scanning the surrounding environment Augmented reality code and its degree of freedom pose relative to the RGB-D camera; and a first recording unit for recording the degree of freedom pose and registering it to the point cloud coordinate system.

Preferably, in the above-mentioned three-dimensional environment modeling device, the one or more augmented reality codes are placed on the surface of the target object.

Preferably, in the above-mentioned three-dimensional environment modeling device, the execution device is configured to construct a geometric structure at the position of the one or more augmented reality codes according to the information carried by the one or more augmented reality codes, The orientation of the geometric structure relative to the one or more augmented reality codes and the content of the geometric structure are encoded in the information.

Preferably, in the above-mentioned three-dimensional environment modeling device, the geometric structure is a polygonal boundary/wall, a rectangular parallelepiped, a cylinder, or the like.

Preferably, in the above-mentioned three-dimensional environment modeling equipment, the execution device is further configured to remove the one or more augmented reality codes from the point cloud stream according to the information carried by the one or more augmented reality codes. Obstacles or boundaries at or near the location of the code, where the extent and orientation of the area of the point to be removed are encoded in the information.

Preferably, in the above-mentioned three-dimensional environment modeling device, the execution device is configured to perform a region operation on the point cloud stream, wherein the region operation includes region segmentation, filtering and/or clustering of the target object , The operating parameters are defined in the information carried by the one or more augmented reality codes.

Another solution of the present invention provides a three-dimensional environment modeling device for an industrial robot, the device comprising: a scanning device for scanning the surrounding environment of the industrial robot to obtain a point cloud flow; and a detection device , Used to detect one or more augmented reality codes in the surrounding environment; execution means, used to be based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes To operate the point cloud stream; and a processing device for processing the operated point cloud stream into a surface mesh, so that the surrounding environment can be modeled.

Another aspect of the present invention provides a computer storage medium, which includes instructions, which execute the aforementioned three-dimensional environment modeling method during runtime.

Another solution of the present invention provides an industrial robot operating platform, which includes the three-dimensional environment modeling device as described above.

Compared with the existing 3D environment modeling solution, the aforementioned 3D environment modeling solution utilizes one or more augmented reality codes in the environment, avoiding or reducing the manual operation involved in the data processing process after the environment scan, and it is more Convenient and efficient. The information carried by the augmented reality code can be customized as needed to modify or perform other operations on the point cloud or grid to be reconstructed.

【Explanation of the drawings】

With reference to the drawings, the disclosure of the present invention will become easier to understand. It is easily understood by those skilled in the art that these drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present invention. In the picture:

Fig. 1 shows a three-dimensional environment modeling method according to an embodiment of the present invention;

Figure 2 shows a three-dimensional environment modeling method for industrial robots according to an embodiment of the present invention; and

Fig. 3 shows a three-dimensional environment modeling device according to an embodiment of the present invention.

【Detailed ways】

The following description describes specific embodiments of the present invention to teach those skilled in the art how to make and use the best mode of the present invention. In order to teach the principles of the invention, some conventional aspects have been simplified or omitted. Those skilled in the art should understand that variations derived from these embodiments will fall within the scope of the present invention. Those skilled in the art should understand that the following features can be joined in various ways to form many variations of the present invention. Therefore, the present invention is not limited to the following specific embodiments, but only by the claims and their equivalents.

Referring to FIG. 1, FIG. 1 shows a three-dimensional environment modeling method 1000 according to an embodiment of the present invention.

In step S110, the surrounding environment is scanned to obtain a point cloud flow.

In step S120, one or more augmented reality codes in the surrounding environment are detected.

In step S130, the point cloud stream is operated based on at least the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes, so as to construct the surrounding environment. mold.

In the context of the present invention, the term "augmented reality code" refers to a mark that performs a specific operation on the point cloud stream obtained after scanning. The augmented reality code can take the form of a two-dimensional code, a data matrix, or a Maxicode, which is not limited in the present invention. The operations performed on the point cloud stream can be encoded or defined in the augmented reality code. In actual use, one or more augmented reality codes can be placed in the work area, such as on the plane and the object to be divided. As an example, these augmented reality codes may carry the following information: whether adjacent surfaces should be included in the reconstructed model or deleted from the reconstructed model, or whether certain point cloud operations (such as segmentation) should be performed. Moreover, the parameters of these operations can be stored in the augmented reality code. As yet another example, augmented reality codes can be used to mark surfaces that cannot be detected by a 3D scanner (such as an RGB-D camera). For example, the augmented reality code can define a glass wall, including its dimensions, or it can add an artificial wall to the reconstruction model where the augmented reality code is placed.

The above-mentioned three-dimensional environment modeling solution utilizes one or more augmented reality codes in the environment, avoids or reduces manual operations involved in the data processing process after environment scanning, and is more convenient and efficient.

In one embodiment, step S110 includes: using an RGB-D camera to scan the surrounding environment from multiple angles. Currently, RGB-D depth cameras include Microsoft Kinect, ASUS Xtion, Obi Zhongguang, Intel RealSense, etc., which are not limited by the present invention. In some embodiments, multiple RGB-D cameras may be used to scan the surrounding environment from multiple angles, and multiple RGB-D cameras may be located at different positions, for example, the first RGB-D camera is located at the first position , The second RGB-D camera is located at a second position different from the first position.

Although not shown in FIG. 1, in one embodiment, the three-dimensional environment modeling method 1000 may further include: recording the point cloud flow. The recording step can be located between step S110 and step S120.

In one embodiment, step S120 includes: when scanning the surrounding environment, using an augmented reality tracking library to detect one or more augmented reality codes in the surrounding environment and their freedom relative to the RGB-D camera Degree pose; and record the degree of freedom pose and register it to the point cloud coordinate system. For example, in step S120, the augmented reality code (such as a two-dimensional code) and its six degrees of freedom pose (and/or other stored information in the augmented reality tracking library) are detected, and they are jointly registered or calibrated to a point The coordinate system of the cloud flow.

In one embodiment, the one or more augmented reality codes are stationary relative to the camera (eg, RGB-D depth camera), which is placed on the surface of the target object. For example, one or more augmented reality codes can be placed on the glass wall to help the RGB-D depth camera recognize the wall. The one or more augmented reality codes are also encoded with the dimensions of the wall surface. For another example, one or more augmented reality codes may be placed at the boundary of the plane, so as to fill a boundary of a predetermined size in the point cloud stream at the position of the one or more augmented reality codes. For another example, one or more augmented reality codes may be placed on the target object, so as to remove the target object or a part of it from the point cloud stream.

In some embodiments, the position and pattern of the augmented reality code may contain information related to customized operations during post-processing of the recorded point cloud stream. In one embodiment, step S130 includes: constructing a geometric structure at the location of the one or more augmented reality codes according to the information carried by the one or more augmented reality codes, wherein the geometric structure is relative to the one or more augmented reality codes. The orientation of the one or more augmented reality codes and the content of the geometric structure are encoded in the information. The geometric structure may be a polygonal boundary/wall, a rectangular parallelepiped or a cylinder, etc., which is not limited in the present invention. In another embodiment, step S130 includes: removing obstacles at or near the location of the one or more augmented reality codes in the point cloud stream according to the information carried by the one or more augmented reality codes The object or boundary in which the extent and orientation of the area of the point to be removed is encoded in the information. In another embodiment, step S130 includes: performing a region operation on the point cloud stream, wherein the region operation includes performing region segmentation, filtering, and/or clustering on the target object, and the parameters of the operation are defined in the point cloud stream. In the information carried by one or more augmented reality codes. Those skilled in the art can understand that the above different embodiments for step S130 can be permuted and combined as needed. For example, step S130 may delete the pattern of points and replace them with other points, that is, simultaneously perform (1) Remove the one or more points from the point cloud stream according to the information carried by the one or more augmented reality codes. Obstacles or borders at or near the location of an augmented reality code, where the range and orientation of the area to be removed are encoded in the information; and (2) according to the one or more augmented reality codes carried Information, construct a geometric structure at the location of the one or more augmented reality codes, wherein the orientation of the geometric structure relative to the one or more augmented reality codes and the content of the geometric structure are encoded in the Information.

Fig. 2 shows a three-dimensional environment modeling method 2000 for an industrial robot according to an embodiment of the present invention.

In step S210, the surrounding environment of the industrial robot is scanned to obtain a point cloud stream.

In step S220, one or more augmented reality codes in the surrounding environment are detected.

In step S230, the point cloud stream is operated at least based on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes.

In step S240, the point cloud stream after the operation is processed into a surface mesh, so that the surrounding environment can be modeled.

In one embodiment, the point cloud stream is acquired and recorded by a handheld RGB-D camera driven by a portable microcomputer. When the area to be recorded and reconstructed is scanned by the RGB-D camera, the augmented reality code is detected by the RGB sensor in the camera. This can be achieved, for example, by including a tracking library (such as ARuCO). The library outputs the 6-DOF pose of the augmented reality code relative to the camera. All the augmented reality code poses will be recorded and co-registered to the recorded point cloud coordinate system.

In one embodiment, after the recording session is completed, the recorded point cloud is streamed to a dedicated PC for offline post-processing, or post-processing on a scanning PC. Specifically, the point cloud stream is input into the latest reconstruction algorithm. In one embodiment, first, the point cloud library (PCL) is used to remove noise from the point cloud stream. Then, PCL and iterative closest point (ICP) algorithm are used to register and stitch the point cloud data to reconstruct the entire environment. Those skilled in the art understand that various iterative nearest point methods ICP can be used, including but not limited to the point-to-plane search for nearest point precise registration method proposed by Chen and Medioni and Bergevin, and the point proposed by Rusinkiewicz and Levoy. -to-projection fast registration method for searching nearby points, Contractive-projection-point registration method for searching nearby points proposed by Soon-Yong and Murali, etc.

Then, using the custom software code (included in the augmented reality code), the location of each detected augmented reality code and the information of each code are imported into the processing sequence. According to the stored information, part of the point cloud stream will be deleted or the defined area will be added to the point cloud, or specific operations will be applied to the point cloud stream. After performing a predefined or customized action or operation, the finally obtained enhanced point cloud stream will be further processed into a surface mesh.

The above-mentioned 3D environment modeling method 2000 realizes the automatic construction of boundaries or shapes in the 3D model by simply adding augmented reality codes to the environment, using the augmented reality codes to automatically remove unwanted surfaces, define reflective surfaces and eliminate noise and 3D starting points Description (for example, filtering, segmentation and other operations on the point cloud).

Fig. 3 shows a three-dimensional environment modeling device 3000 according to an embodiment of the present invention.

As shown in FIG. 3, the three-dimensional environment modeling equipment 3000 includes: a scanning device 310, a detection device 320 and an execution device 330. Wherein, the scanning device 310 is used to scan the surrounding environment to obtain a point cloud stream. The detection device 320 is used to detect one or more augmented reality codes in the surrounding environment. The execution device 330 is configured to operate the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes, so as to construct the surrounding environment. mold.

The above-mentioned three-dimensional environment modeling device 3000 utilizes one or more augmented reality codes in the environment to avoid or reduce manual operations involved in the data processing process after environment scanning, which is more convenient and efficient.

In one embodiment, the scanning device 310 is configured to use an RGB-D camera to scan the surrounding environment from multiple angles. Currently, RGB-D cameras include Microsoft Kinect, ASUS Xtion, Obi Zhongguang, Intel RealSense, etc., which are not limited by the present invention. In some embodiments, the scanning device 310 may use multiple RGB-D cameras to scan the surrounding environment from multiple angles, and the multiple RGB-D cameras may be located at different positions, for example, the first RGB-D camera is located at In the first position, the second RGB-D camera is located in a second position different from the first position.

Although not shown in Fig. 3, the above-mentioned three-dimensional environment modeling device 3000 may further include: a recording device for recording the point cloud stream.

In an embodiment, the detection device 320 may include: a first detection unit and a first recording unit. Wherein, the first detection unit is configured to use an augmented reality tracking library to detect one or more augmented reality codes in the surrounding environment and their degrees of freedom relative to the RGB-D camera when scanning the surrounding environment. posture. The first recording unit is used to record the degree of freedom pose and register it to the point cloud coordinate system. In some embodiments, the detection device 320 may be configured to detect an augmented reality code (such as a two-dimensional code) and its six-degree-of-freedom pose (and/or other stored information in the augmented reality tracking library), and configure them together. Accurate or calibrate to the coordinate system of the point cloud stream.

In some embodiments, the position and pattern of the augmented reality code may contain information related to customized operations during post-processing for the recorded point cloud stream.

In one embodiment, the execution device 330 is configured to construct a geometric structure at the location of the one or more augmented reality codes according to the information carried by the one or more augmented reality codes, wherein the geometric structure is relative to The orientation of the one or more augmented reality codes and the content of the geometric structure are encoded in the information. The geometric structure may be a polygonal boundary/wall, a rectangular parallelepiped or a cylinder, etc., which is not limited in the present invention.

In another embodiment, the execution device 330 is further configured to remove the one or more augmented reality codes in the point cloud stream according to the information carried by the one or more augmented reality codes. Or nearby obstacles or boundaries, where the extent and orientation of the area of the point to be removed is encoded in the information.

In yet another embodiment, the execution device 330 is configured to perform a region operation on the point cloud stream, wherein the region operation includes region segmentation, filtering and/or clustering of the target object, and the parameters of the operation Defined in the information carried by the one or more augmented reality codes.

Those skilled in the art can understand that different embodiments of the execution device 330 described above can be permuted and combined as required. For example, the execution device 330 may be configured to delete the pattern of points and replace them with other points, that is, perform the following two actions at the same time: (1) According to the information carried by the one or more augmented reality codes, in the point cloud Obstacles or boundaries at or near the location where the one or more augmented reality codes are removed in the stream, where the range and orientation of the area of the point to be removed are encoded in the information; and (2) according to the one Information carried by one or more augmented reality codes, a geometric structure is constructed at the location of the one or more augmented reality codes, wherein the geometric structure is relative to the orientation of the one or more augmented reality codes and the geometry The content of the structure is encoded in the information.

In some embodiments, for example, in a scenario where a three-dimensional environment modeling is applied to an industrial robot, the three-dimensional environment modeling device may include the following devices: a scanning device, a detection device, an execution device, and a processing device. Specifically, the scanning device is used to scan the surrounding environment of the industrial robot, so as to obtain a point cloud stream. The detection device is used to detect one or more augmented reality codes in the surrounding environment. The execution device is configured to operate the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes. The processing device is used to process the operated point cloud stream into a surface grid, so that the surrounding environment can be modeled.

Those skilled in the art can easily understand that the three-dimensional environment modeling method provided by one or more embodiments of the present invention can be implemented by a computer program. For example, when a computer storage medium (such as a USB flash drive) storing the computer program is connected to the computer, running the computer program can execute the three-dimensional environment modeling method of the embodiment of the present invention.

In summary, multiple embodiments of the present invention provide a three-dimensional environment modeling solution that uses augmented reality code to accelerate environment reconstruction, and automatically completes and optimizes the point cloud stream containing the augmented reality code part. Although only some of the specific embodiments of the present invention have been described, those of ordinary skill in the art should understand that the present invention can be implemented in many other forms without departing from its spirit and scope, such as being implemented on an industrial robot operating platform. . Therefore, the examples and implementations shown are regarded as illustrative rather than restrictive. The present invention may cover various modifications without departing from the spirit and scope of the present invention as defined by the appended claims. And replace.

Claims

A three-dimensional environment modeling method, the method includes:

Scan the surrounding environment in order to obtain the point cloud flow;

Detect one or more augmented reality codes in the surrounding environment; and

The point cloud stream is operated based on at least the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes, so as to model the surrounding environment.
The 3D environment modeling method according to claim 1, wherein scanning the surrounding environment comprises:

The RGB-D camera is used to scan the surrounding environment from multiple angles.
The three-dimensional environment modeling method according to claim 1, further comprising:

Record the point cloud flow.
The 3D environment modeling method of claim 2, wherein detecting one or more augmented reality codes in the surrounding environment comprises:

When scanning the surrounding environment, use an augmented reality tracking library to detect one or more augmented reality codes in the surrounding environment and their degrees of freedom pose relative to the RGB-D camera; and

Record the pose of the degree of freedom and register it to the point cloud coordinate system.
The three-dimensional environment modeling method of claim 1, wherein the one or more augmented reality codes are placed on the surface of the target object.
The 3D environment modeling method of claim 1, wherein the point cloud stream is performed at least based on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes. Operations include:

According to the information carried by the one or more augmented reality codes, a geometric structure is constructed at the position of the one or more augmented reality codes, wherein the orientation of the geometric structure with respect to the one or more augmented reality codes And the content of the geometric structure is encoded in the information.
The 3D environment modeling method according to claim 6, wherein the geometric structure is a polygonal boundary/wall, a rectangular parallelepiped, or a cylinder.
The three-dimensional environment modeling method of claim 1 or 6, wherein the point cloud is calculated based on at least the information carried by the one or more augmented reality codes and the position of the one or more augmented reality codes. Flow operations include:

According to the information carried by the one or more augmented reality codes, remove obstacles or boundaries at or near the location of the one or more augmented reality codes in the point cloud stream, and the area of the point to be removed The range and orientation of is encoded in the information.
The three-dimensional environment modeling method of claim 5, wherein the point cloud stream is performed at least based on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes. Operations include:

Perform region operations on the point cloud stream, where the region operations include region segmentation, filtering and/or clustering of the target object, and the parameters of the operation are defined in the one or more augmented reality codes carried Information.
A three-dimensional environment modeling method, the method includes:

Scan the surrounding environment of the industrial robot to obtain a point cloud flow;

Detecting one or more augmented reality codes in the surrounding environment;

Operating the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes; and

The point cloud stream after the operation is processed into a surface grid, so that the surrounding environment can be modeled.
A three-dimensional environment modeling device, which includes:

Scanning device, used to scan the surrounding environment in order to obtain the point cloud flow;

A detection device for detecting one or more augmented reality codes in the surrounding environment; and

The execution device is configured to operate the point cloud stream based on at least the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes, so as to construct the surrounding environment mold.
The three-dimensional environment modeling device of claim 11, wherein the scanning device is configured to scan the surrounding environment from multiple angles using an RGB-D camera.
The three-dimensional environment modeling device according to claim 11, further comprising:

The recording device is used to record the point cloud stream.
The three-dimensional environment modeling device according to claim 12, wherein the detection device comprises:

The first detection unit is configured to use an augmented reality tracking library to detect one or more augmented reality codes in the surrounding environment and their degree of freedom pose relative to the RGB-D camera when scanning the surrounding environment ;as well as

The first recording unit is used to record the degree of freedom pose and register it to the point cloud coordinate system.
The three-dimensional environment modeling device of claim 11, wherein the one or more augmented reality codes are placed on the surface of the target object.
The three-dimensional environment modeling device according to claim 11, wherein the execution device is configured to construct at the location of the one or more augmented reality codes according to the information carried by the one or more augmented reality codes A geometric structure, wherein the orientation of the geometric structure relative to the one or more augmented reality codes and the content of the geometric structure are encoded in the information.
The three-dimensional environment modeling device of claim 16, wherein the geometric structure is a polygonal boundary/wall, a rectangular parallelepiped, or a cylinder.
The three-dimensional environment modeling device according to claim 11 or 16, wherein the execution device is further configured to remove the one from the point cloud stream according to the information carried by the one or more augmented reality codes. Obstacles or boundaries at or near the location of or a plurality of augmented reality codes, where the range and orientation of the area of the point to be removed are encoded in the information.
The three-dimensional environment modeling device according to claim 15, wherein the execution device is configured to perform a region operation on the point cloud stream, wherein the region operation includes region segmentation, filtering, and/or the target object Or clustering, the operating parameters are defined in the information carried by the one or more augmented reality codes.
A three-dimensional environment modeling device for industrial robots, the device comprising:

A scanning device for scanning the surrounding environment of the industrial robot so as to obtain a point cloud flow;

A detection device for detecting one or more augmented reality codes in the surrounding environment;

An execution device, configured to operate the point cloud stream based at least on the information carried by the one or more augmented reality codes and the location of the one or more augmented reality codes; and

The processing device is used to process the point cloud stream after the operation into a surface grid, so that the surrounding environment can be modeled.
A computer storage medium comprising instructions that execute the three-dimensional environment modeling method according to any one of claims 1 to 10 when the instructions are run.
An industrial robot operating platform, which comprises the three-dimensional environment modeling equipment according to any one of claims 11 to 20.