WO2023001251A1 - Dynamic picture-based 3d point cloud processing method and apparatus, device and medium - Google Patents

Abstract

Disclosed are a dynamic picture-based 3D point cloud processing method and apparatus, a device and a medium, the method comprising: extracting a region of interest from a scene scanning image of a current scene, dividing the region of interest into a plurality of blocks, and determining a laser scanning range corresponding to each block; for each block, configuring, according to the laser scanning range corresponding to the block, a laser scanning parameter corresponding to the block, and performing laser scanning on the block according to the laser scanning parameter to obtain a 3D point cloud of the block; and splicing 3D point clouds of the plurality of blocks to obtain 3D point clouds of the region of interest. According to the solution, part of the range is cut from the total laser scanning range of a laser scanning device to serve as a laser scanning range corresponding to each block, and laser scanning is performed on each block, thus effectively improving the signal-to-noise ratio, 3D point clouds of the region of interest may be conveniently obtained, and thus effectively improving the accuracy of the 3D point clouds.

Description

3D point cloud processing method, device, equipment and medium based on dynamic frame

This application requires submission of a Chinese patent application with the China Patent Office on July 22, 2021, with the application number 202110831254.8, and the application name "Blocking Processing Method, Device, Computing Equipment, and Storage Medium", and submitted on July 22, 2021 China Patent Office, the application number is 202110832305.9, and the application name is "Image Scanning Method and Device Based on Dynamic Scanning Parameters", and it was submitted to the China Patent Office on July 22, 2021, the application number is 202110832563.7, and the application name is " 3D Point Cloud Processing Method and Device Based on Dynamic Frame”, the priority of the Chinese patent application, the entire content of which is incorporated in this application by reference.

technical field

The present invention relates to the technical field of laser scanning, in particular to a dynamic frame-based 3D point cloud processing method, device, equipment and medium.

Background technique

With the development of industrial intelligence, it is becoming more and more popular to use robots instead of humans to operate objects (such as industrial parts, boxes, etc.). When the robot is operating, it is necessary to use the 3D point cloud corresponding to the current scene as a basis to determine the position of the grasping point of the object. For the 3D point cloud, it is usually obtained by scanning the current scene with a laser scanning device, etc., and then processing the scanned image. However, in the process of laser scanning, it is easy to be disturbed by ambient light, reflected light on the surface of the measured object, etc., resulting in poor image quality parameters such as signal-to-noise ratio of the scanned image, which in turn affects the accuracy of the 3D point cloud.

technical solution

In view of the above problems, the present invention is proposed to provide a dynamic frame-based 3D point cloud processing method, device, device and medium that overcomes the above problems or at least partially solves the above problems.

According to one aspect of the present invention, a kind of 3D point cloud processing method based on dynamic frame is provided, and the method comprises:

Extract the region of interest from the scene scan image of the current scene, divide the region of interest into multiple blocks, and determine the laser scanning range corresponding to each block;

For each block, according to the laser scanning range corresponding to the block, configure the laser scanning parameters corresponding to the block, and perform laser scanning on the block according to the laser scanning parameters to obtain the 3D point cloud of the block;

The 3D point cloud of multiple blocks is spliced to obtain the 3D point cloud of the region of interest.

According to another aspect of the present invention, a dynamic frame-based 3D point cloud processing device is provided, the device comprising:

The block module is adapted to extract the region of interest from the scene scan image of the current scene, divide the region of interest into multiple blocks, and determine the laser scanning range corresponding to each block;

The scanning module is adapted to configure laser scanning parameters corresponding to each block according to the laser scanning range corresponding to the block, and perform laser scanning on the block according to the laser scanning parameters to obtain 3D points of the block cloud;

The splicing module is suitable for splicing 3D point clouds of multiple blocks to obtain a 3D point cloud of the region of interest.

According to yet another aspect of the present invention, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete mutual communication through the communication bus;

The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above dynamic frame-based 3D point cloud processing method.

According to yet another aspect of the present invention, a computer storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform operations corresponding to the above-mentioned dynamic frame-based 3D point cloud processing method.

According to the 3D point cloud processing method, device, device and medium based on the dynamic frame provided by the present invention, the region of interest in the scene scanning image is divided into multiple blocks, and a part is intercepted from the total laser scanning range of the laser scanning device The range is used as the laser scanning range corresponding to each block to realize a dynamic frame; according to the laser scanning range corresponding to each block, configure the laser scanning parameters corresponding to each block, and perform laser scanning on the block according to the laser scanning parameters. Concentrating the laser energy per unit time helps to obtain a better laser scanning effect and effectively improves the signal-to-noise ratio; by splicing multiple 3D point clouds of blocks, the 3D image of the region of interest can be easily obtained Point cloud, and effectively improve the accuracy of 3D point cloud, improve the quality of point cloud, and optimize the point cloud processing method.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Fig. 1 shows an application scenario diagram of a dynamic frame-based 3D point cloud processing method provided by an embodiment of the present invention;

Fig. 2 shows a schematic flow diagram of a 3D point cloud processing method based on a dynamic frame according to an embodiment of the present invention;

Fig. 3a shows a schematic flow chart of a dynamic frame-based 3D point cloud processing method according to yet another embodiment of the present invention;

Fig. 3b shows a flow chart of image quality model determination in the embodiment shown in Fig. 3a;

Fig. 4a shows a schematic flow chart of a dynamic frame-based 3D point cloud processing method according to yet another embodiment of the present invention;

Fig. 4b shows the flow chart of determining the blocking parameters in the embodiment shown in Fig. 4a;

Fig. 4c shows a schematic diagram of the region of interest in the current scene image in the embodiment shown in Fig. 4a;

FIG. 5 shows a schematic flow diagram of a dynamic frame-based 3D point cloud processing method according to yet another embodiment of the present invention;

Fig. 6 shows a structural block diagram of a 3D point cloud processing device based on a dynamic frame according to an embodiment of the present invention;

Fig. 7 shows a schematic structural diagram of a computing device according to an embodiment of the present invention.

Embodiments of the present invention

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

In the logistics and warehousing field and other scenes that require unmanned picking or grabbing of objects, it is necessary to use the 3D point cloud corresponding to the current scene as the basis to determine the grabbing point position of the object. The accuracy of grab point location determination is based on the accuracy of the 3D point cloud, and the 3D point cloud is usually obtained by scanning the current scene with a laser scanning device, etc., and then processing the scanned image. However, in the process of laser scanning, it is easy to be disturbed by ambient light, reflected light on the surface of the measured object, etc., resulting in poor image quality parameters such as signal-to-noise ratio of the scanned image, and the image quality of each image position in the scanned image is not good. Evenly, the image quality of some areas of the scanned image is often poor, which leads to poor accuracy of the 3D point cloud and is difficult to improve.

In order to solve this problem, an embodiment of the present disclosure provides a dynamic frame-based 3D point cloud processing method, which divides the region of interest in the scene scan image into blocks, and determines the region of interest according to the laser scanning parameters of each block. The block point cloud, and then through the splicing of the point cloud, the overall point cloud is obtained, so as to ensure the accuracy of each block 3D point cloud, and then ensure the overall accuracy of the 3D point cloud.

The application scenarios of the embodiments of the present disclosure are explained below:

FIG. 1 is an application scene diagram of a dynamic frame-based 3D point cloud processing method provided by an embodiment of the present disclosure. As shown in FIG. 1 , in the process of acquiring a 3D point cloud, a laser scanning device 100 captures a current scene 110 and transmits the data to a processor 120 to generate 3D point cloud data of a region of interest 130 in the current scene 110 . Fig. 2 shows a schematic flow chart of a 3D point cloud processing method based on a dynamic frame according to an embodiment of the present invention. As shown in Fig. 2, the method includes the following steps:

Step S201, extracting the region of interest from the scene scan image of the current scene, dividing the region of interest into multiple blocks, and determining the laser scanning range corresponding to each block.

Specifically, the scene scan image of the current scene can be obtained by an image acquisition device, and the scene scan image is a 2D image or a 3D image obtained based on the current scene for extracting 3D point cloud data, which is not specifically limited in this solution.

After the scene scan image is obtained, its corresponding image quality parameters, such as contrast and signal-to-noise ratio, can be determined based on the scene scan image. If the image quality parameters of the scene-scanned image are relatively good and it is a 3D image, the 3D point cloud can be obtained directly from the scene-scanned image, and there is no need to process the 3D point cloud based on the dynamic frame.

If the scene scan image is poor, the accuracy of the 3D point cloud obtained directly from the scene scan image cannot be guaranteed, and further processing is required.

Among them, the area of interest is the area where the object that needs to generate 3D point cloud data is located. For example, if the current scene needs to scan pallets, baskets, cage cars and other stacked containers, then the area corresponding to the stacked container in the scene image is taken as the area of interest. area.

Since the region of interest may have uneven image quality (or different image quality parameters) in various parts, it is necessary to divide the region of interest into multiple blocks to determine the corresponding image quality parameters according to different blocks. Laser scanning parameters (such as determining the laser scanning speed according to the image signal-to-noise ratio, and determining the laser scanning range according to the range of each block), in order to maximize the image quality obtained by laser scanning of each block (and make each block stable image quality parameters), in order to generate a 3D point cloud based on the images obtained by each block, and ensure the accuracy of the 3D point cloud.

Step S202, for each block, configure laser scanning parameters corresponding to the block according to the laser scanning range corresponding to the block.

Specifically, after the laser scanning range corresponding to each block is determined, the laser scanning parameters corresponding to the block can be configured for each block according to the laser scanning range corresponding to the block.

Laser scan parameters may include laser scan speed. The larger the laser scanning range, the slower the laser scanning speed can be to ensure the high quality of the scanned image.

In some embodiments, in order to obtain a better scanning effect, a slower laser scanning speed can be used for scanning, so as to concentrate the laser energy per unit time and improve the signal-to-noise ratio.

Step S203, performing laser scanning on the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block.

Specifically, after the laser scanning parameters are determined, the image acquisition device can be controlled to scan the corresponding block based on the laser scanning parameters to obtain the 3D image corresponding to the block. A corresponding 3D point cloud can be obtained based on the segmented image.

The 3D point cloud includes pose information of each 3D point, and the pose information of each 3D point may specifically include information such as the coordinate value of each 3D point in the XYZ three-axis in space and the XYZ three-axis direction of each 3D point itself. Obtaining a 3D point cloud from a 3D image is a conventional technical means in this field, and is not the focus of this solution, so it will not be repeated here.

In step S204, the 3D point clouds of the multiple blocks are spliced to obtain the 3D point clouds of the region of interest.

Specifically, since in this embodiment, laser scanning is performed on each sub-block separately, what is obtained is a 3D point cloud of each sub-block, not a complete 3D point cloud of the region of interest, so multiple The segmented 3D point cloud is spliced to obtain the 3D point cloud of the region of interest.

Considering that the block is divided according to the overlap rate in the block process (that is, a certain overlap rate/a part of the overlapping area is allowed between the blocks), so that there is an overlapping area in the 3D point cloud of two adjacent blocks, That is to say, there are two sets of 3D point clouds corresponding to the overlapping area, and a set of 3D point clouds with better point cloud quality needs to be selected from the two sets of 3D point clouds for splicing.

The point cloud quality may include the signal-to-noise ratio of the point cloud. By comparing the point cloud quality of each block in the overlapping area, the 3D point cloud for splicing can be selected, and then the 3D point cloud of the region of interest can be obtained through splicing.

According to the dynamic frame-based 3D point cloud processing method provided in this embodiment, the region of interest in the scene scanning image is divided into multiple blocks, and a part of the range is intercepted from the total range of laser scanning of the laser scanning device as the corresponding block. According to the laser scanning range corresponding to each block, configure the laser scanning parameters corresponding to each block, and perform laser scanning on the block according to the laser scanning parameters, so that the laser energy per unit time Concentration helps to obtain a better laser scanning effect and effectively improves the signal-to-noise ratio; by splicing multiple 3D point clouds of blocks, the 3D point cloud of the region of interest can be easily obtained, and effectively The accuracy of 3D point cloud is improved, the quality of point cloud is improved, and the processing method of point cloud is optimized.

Fig. 3a shows a schematic flow chart of a method for processing a 3D point cloud based on a dynamic frame according to yet another embodiment of the present invention. As shown in Fig. 3a, the method includes the following steps:

Step S301, according to the image quality model and target image quality parameters, determine the laser scanning parameters corresponding to each laser scanning position.

Wherein, the image quality model is used to indicate the corresponding relationship between the laser scanning parameters and the image quality parameters corresponding to each laser scanning position.

Specifically, the image quality model is a model obtained by training an initial model with laser scanning parameters corresponding to the sample scanned image and image quality parameters corresponding to each laser scanning position in the sample scanned image as sample data. The initial model can be a neural network model or other models (such as a ghost function model, a logarithmic function model, an exponential function model, a hyperbolic function model, etc.), and a trained image quality model is obtained.

After the trained image quality model is obtained, the image quality model can be used to determine laser scanning parameters corresponding to each laser scanning position during a complete laser scanning process.

Specifically, the target image quality parameter can be set according to the scanning requirements, and the target image quality parameter is the image quality parameter that can be achieved by the scanned image of the expected scene, such as the expected signal-to-noise ratio; since the image quality model includes the laser scanning position The corresponding relationship between the corresponding laser scanning parameters and the image quality parameters, then according to the target image quality parameters and the corresponding relationship, the laser scanning parameters corresponding to each laser scanning position can be accurately determined. Through this processing method, accurate determination of dynamic laser scanning parameters is conveniently realized.

In some embodiments, the laser scanning parameters include any of the following: laser signal intensity or laser scanning speed.

Specifically, the laser scanning parameters may also include other parameters, which are not limited here. That is to say, during a complete laser scanning process, the laser scanning parameters are fixed values, for example, laser scanning is performed according to a fixed laser signal intensity and a fixed laser scanning speed.

In some embodiments, as shown in Fig. 3b, it is a flow chart of image quality model determination. The image quality model is obtained as follows:

Step S3011, performing multiple laser scans on the sample scene to obtain multiple sample scan images.

Wherein, different sample scanning images correspond to different laser scanning parameters.

Specifically, use laser scanning equipment (that is, image acquisition equipment) to perform laser scanning on the scene. The laser scanning equipment can specifically be a 3D laser camera (or two 2D laser cameras with a fixed angle), and the laser scanning equipment can be set at the upper position The location, such as directly above or obliquely above, is used to scan the scene information of the scene. In the existing laser scanning process, the laser scanning equipment is usually controlled to work according to preset and fixed laser scanning parameters, and the laser is used to perform laser scanning on the scene.

However, due to the influence of interference light, equipment assembly of the laser scanning device itself, etc., the image quality of the scanned image obtained by scanning is not uniform enough, and the image quality of the middle area is generally better than that of the edge area.

Considering that there is a potential correlation between laser scanning position, laser scanning parameters and image quality, in order to obtain the correlation, in the present invention, a number of different laser scanning parameters are preset, and the sample scene of laser scanning is selected . In order to make the trained image quality model more applicable to the scanning scene in the actual application, the scanning scene in the actual application or the scene similar to the scanning scene in the actual application can be selected as the sample scene, for example, in the application of robot depalletizing and palletizing , you can choose a scene containing stacked containers such as pallets, baskets, and cage cars as a sample scene.

Then, the laser scanning device is used to perform multiple laser scans on the sample scene according to multiple laser scanning parameters to obtain multiple sample scanning images, wherein different sample scanning images correspond to different laser scanning parameters. That is to say, according to the first laser scanning parameter, a complete laser scanning is performed on the sample scene to obtain the first sample scanning image, and the first sample scanning image corresponds to the first laser scanning parameter; and then according to the first Two laser scan parameters, a complete laser scan of the sample scene is performed to obtain the second sample scan image; by analogy, multiple sample scan images are obtained.

Step S3012, image analysis is performed on the multiple scanned images of the samples to obtain the image quality parameters of each image position in the multiple scanned images of the samples.

Specifically, after obtaining multiple scanned images of samples, image analysis is performed on the scanned images of multiple samples. Specifically, the edge clarity of each image position in the scanned image of each sample, the absence of midpoints in the image, etc. can be analyzed, so as to obtain Image quality parameters for each image location in each sample scan image.

Wherein, the image quality parameter may include at least one of contrast, signal-to-noise ratio, edge sharpness, average brightness, histogram, and the like.

Step S3013, using the image quality parameters of each image position in the multiple sample scan images, the laser scan parameters corresponding to the multiple sample scan images, and the predetermined correspondence between the image position and the laser scan position for training to obtain an image quality model .

Specifically, in order to facilitate training, it is also necessary to determine the correspondence between the image position and the laser scanning position. In practical applications, during the laser scanning process, the relative positional relationship between the laser scanning position and the scene information of the sample scene can be recorded, and then the relative position between the image position in the sample scanning image obtained by scanning and the scene information of the sample scene can be recorded. Positional relationship, to determine the corresponding relationship between the image position and the laser scanning position.

The image quality model can be obtained by substituting the determined image quality parameters, laser scanning parameters and the corresponding relationship between the image position and the laser scanning position into the initial model for training.

Further, the step of obtaining the image quality model may include:

Step 1 (not shown), according to the image quality parameters of each image position in the multiple sample scanned images and the corresponding relationship between the image position and the laser scanning position, determine the image corresponding to each laser scanning position in the multiple sample scanned images quality parameters.

Specifically, to determine the image quality parameter, the image quality parameter of each image position in the multiple sample scan images can be converted into the image quality parameter of the corresponding laser scanning position according to the correspondence between the image position and the laser scanning position, thereby completing Determination of image quality parameters corresponding to each laser scanning position in multiple sample scanning images.

Step 2 (not shown), sample data is extracted from the laser scanning parameters corresponding to the multiple sample scanning images and the image quality parameters corresponding to the respective laser scanning positions in the multiple sample scanning images.

Wherein, the sample data includes laser scanning parameters, laser scanning positions, and image quality parameters corresponding to the laser scanning parameters and laser scanning positions.

Specifically, the laser scanning position that can determine the corresponding relationship with the image position, the image quality parameter corresponding to the laser scanning position, and the laser scanning parameters of each sample scanning image corresponding to the image position can be used as sample data. for training. If the corresponding relationship between the laser scanning position, image quality parameters and laser scanning parameters is not determined (for example, only the laser scanning parameters are determined, but the corresponding laser scanning position and image quality parameters are not determined), it cannot be used as sample data.

The initial model is trained through sample data. The initial model can be a neural network model or other models (such as a ghost function model, a logarithmic function model, an exponential function model, a hyperbolic function model, etc.), and a trained image quality model is obtained.

Step 3 (not shown), using the sample data to train the initial model to obtain an image quality model.

Specifically, the specific training process can be: input the laser scanning parameters and laser scanning position into the initial model for training, obtain the training output result, calculate the loss between the training output result and the image quality parameter, and obtain the loss function, according to the loss function Update the weight parameters of the initial model; execute the above steps cyclically and iteratively until the iteration end condition is satisfied, and the image quality model is obtained.

During the training process, the input variables of the initial model are the laser scanning parameters and the laser scanning position, and the output is the image quality parameters obtained through training, that is, the training output results. The weight parameters of the initial model can be adjusted by gradient descent method or quasi-Newton method to make the loss function reach the global minimum. Wherein, the iteration end condition may include: the number of iterations reaches a threshold of iterations; and/or, the output value of the loss function is smaller than the threshold of loss. Then, it can be judged whether the iteration end condition is met by judging whether the number of iterations reaches the iteration number threshold, or whether the iteration end condition is met according to whether the output value of the loss function is less than the loss threshold. After the iteration end condition is satisfied, the iterative process is stopped to obtain the image quality model, which is the trained model, and the model includes the corresponding relationship between the laser scanning parameters corresponding to each laser scanning position and the image quality parameters .

Step S302, at each laser scanning position, perform laser scanning on the current scene according to the corresponding laser scanning parameters, to obtain a scene scanning image of the current scene.

Specifically, after determining the laser scanning parameters corresponding to each laser scanning position, at each laser scanning position, the laser scanning device is controlled to work according to the laser scanning parameters corresponding to the laser scanning position, and the current scene is scanned by laser , the output scene scan image.

In some embodiments, during a complete laser scanning process, the laser scanning is not performed according to the fixed laser scanning parameters from the beginning to the end, but at different laser scanning positions, according to the laser scanning parameters corresponding to the laser scanning positions , use the laser scanning device to perform laser scanning on the current scene, that is, perform laser scanning based on dynamic laser scanning parameters, and finally obtain a scene scanning image with relatively uniform image quality, so that the areas in the scene scanning image with poor image quality (such as The image quality of the edge area) is effectively improved.

Step S303, analyzing the scene scan image of the current scene to obtain actual image quality parameters of the scene scan image.

Specifically, the image of the current scene may be collected by an image collection device such as 2D/3D to obtain a scene scan image of the current scene. The scene scan image may be a 2D image or a 3D image, which is not limited here. Considering that if the image quality parameter of the scene scan image is relatively good and it is a 3D image, then the 3D point cloud can be obtained directly according to the scene scan image, and there is no need to perform 3D point cloud processing based on the dynamic frame mode. In this embodiment, The dynamic frame refers to dynamically intercepting part of the range from the total laser scanning range of the laser scanning device as the current laser scanning range, that is, as the laser scanning range corresponding to each block; It is necessary to analyze the scene scan image of the current scene, for example, analyzing the edge clarity in the scene scan image, the lack of midpoint in the image, etc., to obtain the image quality parameters of the scene scan image.

The image quality parameter may include at least one of contrast, signal-to-noise ratio, edge sharpness, average brightness, histogram, and the like.

Step S304, if the actual image quality parameter is less than the preset parameter threshold, extract the region of interest from the scene scan image of the current scene.

Specifically, it can be judged whether the image quality parameter of the scene scanning image is less than the preset parameter threshold; if the image quality parameter is less than the preset parameter threshold, it indicates that the image quality of the scene scanning image is poor, and the 3D point cloud processing is performed based on the dynamic frame method , in combination with this embodiment, the region of interest (Region of Interest, ROI) is extracted from the scene scan image of the current scene; if the image quality parameter is greater than or equal to the preset parameter threshold, it indicates that the image quality of the scene scan image is better, directly based on The 3D point cloud can be obtained from the 3D scene scanning image, and there is no need to process the 3D point cloud based on the dynamic frame, and the method ends.

Wherein, those skilled in the art can set the preset parameter threshold according to actual needs, which is not limited here.

Step S305, dividing the region of interest into multiple blocks, and determining the laser scanning range corresponding to each block.

Step S306, for each block, according to the laser scanning range corresponding to the block, obtain the laser scanning parameters corresponding to the block.

Step S307, performing laser scanning on the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block.

In step S308, the 3D point clouds of the multiple blocks are spliced to obtain the 3D point clouds of the region of interest.

Specifically, the contents of step S305 to step S308 are the same as the corresponding steps in the embodiment shown in FIG. 2 , and will not be repeated here.

According to the dynamic frame-based 3D point cloud processing method provided in this embodiment, firstly, according to the image quality model, the laser scanning parameters corresponding to each laser scanning position in the current scene are accurately determined, and at different laser scanning positions, dynamically according to the corresponding The laser scanning parameters of the current scene are laser scanned, and the laser scanning based on the dynamic laser scanning parameters is realized to ensure that the overall image quality of the scene scanning image obtained by scanning the current scene is relatively uniform, and then the scene scanning image is scanned. The area is divided into multiple blocks, and part of the range is intercepted from the total laser scanning range of the laser scanning device as the laser scanning range corresponding to each block, and the corresponding laser scanning range of each block is configured according to the corresponding laser scanning range of each block. Laser scanning parameters, according to the laser scanning parameters, the block is scanned by laser, and the 3D point cloud of the region of interest can be obtained conveniently by splicing the 3D point clouds of multiple blocks. As a result, the image quality of areas such as edges with poor image quality in the scene scanning image used for analysis is effectively improved, and the image scanning method is optimized. When generating a 3D point cloud based on the region of interest in the scene image, it can be It effectively improves the accuracy of 3D point cloud and improves the quality of point cloud.

Fig. 4a shows a schematic flow chart of a 3D point cloud processing method based on a dynamic frame according to an embodiment of the present invention. As shown in Fig. 4a, the method includes the following steps:

Step S401, extracting a region of interest from a scene scan image of a current scene.

Specifically, for the method for determining the region of interest, reference may be made to the corresponding content in the embodiments shown in FIG. 2 and FIG. 3 , which will not be repeated here.

Step S402, according to the setting parameters of the laser scanning device, determine the laser scanning range corresponding to the region of interest.

Specifically, after the region of interest is determined, the laser scanning range corresponding to the region of interest may be determined according to the setting parameters of the laser scanning device. The setting parameters of the laser scanning device include the setting position of the laser scanning device, the total range of the laser scanning and other parameters. Wherein, the laser scanning device may be arranged at an upper position, such as directly above or obliquely above, for scanning information of the current scene. Specifically, the laser scanning range corresponding to the region of interest may be determined according to the position information of the region of interest in the scene scanning image and the setting parameters of the laser scanning device. The laser scanning range corresponding to the region of interest is smaller than the total laser scanning range, and the laser scanning range can be specifically represented by a laser scanning angle range.

Step S403, acquiring block parameters, dividing the region of interest into multiple blocks according to the block parameters, and recording the position information of each block in the region of interest.

Wherein, the block parameters include: the number of blocks and the overlap rate.

Specifically, the block parameter may be preset, or may be automatically calculated according to the image quality parameters of the scanned image of the scene.

In some embodiments, as shown in Fig. 4b, it is a flow chart of determining the block parameters. Obtain block parameters, including the following steps:

Step S4031, acquiring an area image of the area of interest.

Wherein, the region image refers to an image of a region corresponding to the region of interest in the scene scan image of the current scene.

Specifically, the area image can identify the area of interest according to the scanning requirements of the current scene. For example, if the current scene needs to scan stacked containers such as pallets, material baskets, and cage cars, the area corresponding to the stacked containers in the scene image will be taken as the region of interest. Then the region image of the region of interest is obtained from the scene image.

Fig. 4c shows a schematic diagram of the region of interest in the current scene image. As shown in Fig. 4c, the shaded part 42 in the current scene image 41 is a conveyor belt. Interest region, then the area image of the interest region is the image corresponding to the shaded part 12.

Step S4032, analyzing the abnormal points in the regional image to obtain abnormal point data.

Specifically, for the abnormal points existing in the regional image, the abnormal point data is obtained by identifying and analyzing the abnormal points in the regional image. Wherein, the abnormal points include but are not limited to: flying points, empty points, and singular points. Flying points refer to the points that fly out of the image; empty points refer to the vacant points in the image; singular points refer to the points in the image that are too different from the surrounding pixels, which is visually dazzling.

Further, if the outlier data includes the proportion of outliers, the method for determining the outlier data may include:

Identify the abnormal points existing in the regional image, calculate the proportion of the abnormal points in the regional image, and obtain the proportion of the abnormal points.

Specifically, by calculating the proportion of abnormal points in the region image, the proportion of abnormal points can be obtained. For example, the ratio of the number of abnormal points to the number of all points in the region image can be calculated, and the calculation result can be used as the proportion of abnormal points.

Step S4033, using the pre-built evaluation model to process the outlier data and the quality parameters of the target point cloud to obtain the block parameters corresponding to the current scene.

Among them, the evaluation model is used to indicate the corresponding relationship between abnormal point data, point cloud quality parameters and block parameters.

Specifically, after the abnormal point data is obtained, the segmentation parameters corresponding to the current scene can be automatically determined according to the abnormal point data, so as to segment the region of interest according to the segmentation parameters.

In some embodiments, pre-built evaluation models can be utilized to determine the segmentation parameters. Considering that there is a potential correlation between block parameters and image quality, image quality can be reflected by abnormal point data, point cloud quality parameters, etc.

Further, the evaluation model is obtained as follows:

Step 1 (not shown), acquiring collected sample outlier data corresponding to multiple sample area images, sample block parameters corresponding to multiple sample area images, and sample point cloud quality corresponding to multiple sample area images parameter.

Specifically, in order to be able to construct an evaluation model to reflect the correlation, it is necessary to collect sample outlier data corresponding to multiple sample area images, sample block parameters corresponding to multiple sample area images, and data corresponding to multiple sample area images. The corresponding sample point cloud quality parameters are used for training to obtain the evaluation model.

In order to obtain a data-rich sample data set, different sample area images can be divided into blocks using different sample block parameters.

Wherein, the sample point cloud quality parameter may be an average value, variance, etc. of the point cloud quality parameters of multiple block sample 3D point clouds. The calculation methods of sample point cloud quality parameters include:

Obtain multiple sample area images, analyze the abnormal points in multiple sample area images, and obtain sample abnormal point data corresponding to multiple sample area images; for each sample area of interest corresponding to the sample area image, according to the sample The block parameter divides the region of interest of the sample into blocks, obtains the block group corresponding to the sample block parameter, and determines the laser scanning range corresponding to each block in the block group; according to the laser scanning range corresponding to each block, Laser scanning is performed on each block to obtain the sample 3D point cloud of each block; the sample 3D point cloud of multiple blocks is analyzed to obtain the sample point cloud quality parameters corresponding to the image of the sample area.

Specifically, in the training phase, in order to make the constructed evaluation model more applicable to the scanning scene in the actual application, the scanning scene in the actual application or the scene similar to the scanning scene in the actual application can be selected as the sample scene, for example, in the robot dismantling In the application of stacking and stacking, you can choose a scene that includes stacking containers such as pallets, baskets, and cage cars as a sample scene.

Step 2 (not shown), constructing Sample dataset.

Specifically, after determining the sample abnormal point data corresponding to the image of the sample area, the sample block parameters and the sample point cloud quality parameters, they can be used as training samples to train the evaluation model.

Step 3 (not shown), perform training according to the sample data set, and construct an evaluation model.

Specifically, the sample outlier data corresponding to the sample area image, the sample block parameters corresponding to the sample area image, and the sample point cloud quality parameters corresponding to the sample area image are extracted from the sample data set; the sample outlier data and the sample point The cloud quality parameters are input into the initial evaluation model for training, and the initial block results corresponding to the sample area images are obtained; according to the initial block results and the sample block parameters corresponding to the sample area images, the weight parameters of the initial evaluation model are updated; loop Perform the above steps iteratively until the iteration end condition is met, and the evaluation model is obtained.

During the training process, the input variables of the initial evaluation model are sample outlier data and sample point cloud quality parameters, and the output is the block parameters obtained from training, that is, the initial block results. Gradient descent method or quasi-Newton method can be used to adjust the weight parameters of the initial evaluation model to make the loss function reach the global minimum. Wherein, the iteration end condition may include: the number of iterations reaches a threshold of iterations; and/or, the output value of the loss function is smaller than the threshold of loss. Then, it can be judged whether the iteration end condition is met by judging whether the number of iterations reaches the iteration number threshold, or whether the iteration end condition is met according to whether the output value of the loss function is less than the loss threshold. After the iteration end condition is satisfied, the iterative process is stopped to obtain the evaluation model, which is the trained and constructed model, which includes the correspondence between abnormal point data, point cloud quality parameters and block parameters relation.

After completing the construction of the evaluation model, when the region of interest in the current scene needs to be divided into blocks, the abnormal point data in the region of interest and the preset target point cloud quality parameters are input into the evaluation model for processing. The evaluation model outputs the block parameters corresponding to the current scene. Using the evaluation model can accurately, quickly and automatically determine the block parameters corresponding to the current scene, effectively improving the determination accuracy and efficiency of the block parameters.

Step S404, for each block, determine the laser scanning range corresponding to each block according to the position information of the block in the region of interest and the laser scanning range corresponding to the region of interest.

Specifically, the laser scanning range corresponding to each block is smaller than the laser scanning range corresponding to the region of interest, which is equivalent to intercepting a part of the range from the total laser scanning range of the laser scanning device as the laser scanning range corresponding to each block. During scanning, only the information in one block is scanned.

For example, when the number of blocks is 4 and the overlap rate is 5%, it means that the region of interest needs to be divided into 4 blocks, and 5% of the areas between two adjacent blocks overlap. According to the preset direction (such as the direction from left to right), the four blocks are block 1, block 2, block 3, and block 4, and there is a 5% difference between block 1 and block 2. The areas are overlapping, 5% of the areas of block 2 and block 3 are overlapped, and 5% of the areas of block 3 and block 4 are overlapped.

Step S405, for each block, according to the laser scanning range corresponding to the block, obtain the laser scanning parameters corresponding to the block.

Step S406, performing laser scanning on the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block.

Step S407, performing splicing processing on the 3D point clouds of multiple blocks to obtain a 3D point cloud of the region of interest.

Specifically, the contents of step S405 to step S407 are the same as the corresponding steps in the embodiment shown in FIG. 2 , and will not be repeated here.

According to the dynamic frame-based 3D point cloud processing method provided in this embodiment, the region of interest is extracted from the scene scan image of the current scene, and then the laser scanning range corresponding to the region of interest is determined according to the setting parameters of the laser scanning device, and the points are obtained. Block parameters, according to the block parameters, divide the region of interest into multiple blocks, and record the position information of each block in the region of interest, and then for each block, according to the block in the region of interest The location information and the laser scanning range corresponding to the region of interest, determine the laser scanning range corresponding to each block, and then determine the laser scanning parameters according to the laser scanning range in turn, and then determine the 3D point cloud of each block, and finally get the region of interest 3D point cloud. Since the block parameters are determined based on the outlier data corresponding to the region image of the region of interest, the accuracy and efficiency of the block parameter determination are effectively improved; the region of interest is divided into blocks according to the block parameters to achieve a reasonable block , by scanning in blocks and then splicing, the 3D point cloud of the region of interest can be easily obtained, and the accuracy of the 3D point cloud can be effectively improved.

Fig. 5 shows a schematic flow chart of a 3D point cloud processing method based on a dynamic frame according to an embodiment of the present invention. As shown in Fig. 5, the method includes the following steps:

Step S501, extracting the region of interest from the scene scan image of the current scene, dividing the region of interest into multiple blocks, and determining the laser scanning range corresponding to each block.

Step S502, for each block, according to the laser scanning range corresponding to the block, obtain the laser scanning parameters corresponding to the block.

Specifically, for step S501 and step S502, reference may be made to corresponding descriptions in the embodiments shown in FIG. 2 to FIG. 4 , and details are not repeated here.

Step S503, according to the laser scanning parameters, control the rotation of the vibrating mirror in the laser scanning device, and use the laser reflected by the vibrating mirror to perform laser scanning on the block to obtain the 3D point cloud of the block.

Wherein, the laser scanning parameters include any of the following: laser scanning angle range, laser signal intensity or laser scanning speed.

Specifically, the laser scanning device can be the image acquisition device in the aforementioned solution, or other devices, such as a laser light source and a vibrating mirror based on a MEMS (Micro-Electro-Mechanical System, micro-electro-mechanical system) process. The mirror includes a vibrating mirror motor, and the vibrating mirror motor is also connected with a reflecting mirror. The vibrating mirror motor rotates according to the instructions of the laser scanning device, and the rotation of the vibrating mirror motor drives the mirror mirror connected to it to rotate, thereby adjusting the position of the mirror mirror. For each block, the movement of the corresponding galvanometer can be determined according to the laser scanning parameters, so as to obtain the 3D point cloud of the corresponding block.

Step S504, for the 3D point clouds of any two adjacent blocks, according to the position information of the two adjacent blocks in the region of interest, perform intersection processing on the 3D point clouds of the two adjacent blocks to obtain the overlapping Area point clouds as well as non-overlapping area point clouds.

Specifically, the point cloud quality of the point cloud in the overlapping area is analyzed, such as analyzing the point cloud noise ratio, point cloud density, point cloud thickness, and point cloud overlapping degree, etc., to obtain the point cloud quality of the point cloud in the overlapping area. Among them, point cloud noise is gross error, which can be divided into point-like gross error and cluster-like gross error from the spatial distribution; point cloud density refers to the density of laser data points. With the development of laser scanning technology, it can reach Hundreds of points; the point cloud thickness refers to the error of the point cloud elevation in the flat area of the 3D point cloud to be analyzed; the point cloud overlap refers to the convex polygon of the 3D point cloud to be analyzed and the difference between the adjacent point cloud The ratio of the area where the convex polygons of the flight strips intersect to the convex polygon of the flight strips of the 3D point cloud to be evaluated.

Step S505, according to the point cloud quality of the overlapping area point cloud, select the target overlapping area point cloud for splicing from the overlapping area point cloud, and perform splicing processing on the target overlapping area point cloud and the non-overlapping area point cloud.

Specifically, the target overlapping area point cloud is a set of 3D point clouds with better point cloud quality among the two sets of overlapping area point clouds. Through 3D point cloud fusion processing, the stitching of 3D point clouds of each block is realized.

In step S506, the 3D point cloud of the region of interest is obtained.

Specifically, according to the above processing manner, the splicing processing of all the segmented 3D point clouds is completed, so as to obtain the 3D point cloud of the region of interest.

According to the dynamic frame-based 3D point cloud processing method provided in this embodiment, the region of interest in the scene scan image is divided into multiple blocks, and according to the laser scanning range corresponding to each block, the corresponding laser scanning range of each block is configured. Laser scanning parameters, according to the laser scanning parameters, the block is scanned by laser, so that the laser energy is concentrated per unit time, which helps to obtain a better laser scanning effect and effectively improves the signal-to-noise ratio; By splicing 3D point clouds, the 3D point cloud of the region of interest can be easily obtained, and the accuracy of the 3D point cloud is effectively improved, the quality of the point cloud is improved, and the point cloud processing method is optimized.

Fig. 6 shows a structural block diagram of a 3D point cloud processing device based on a dynamic frame according to an embodiment of the present invention. As shown in Fig. 6, the 3D point cloud processing device 600 based on a dynamic frame includes: a block module 610, an acquisition module 620 , scanning module 630 and stitching module 640 .

Blocking module 610, configured to extract the region of interest from the scene scan image of the current scene, divide the region of interest into multiple blocks, and determine the laser scanning range corresponding to each block.

The obtaining module 620 is used for obtaining the laser scanning parameters corresponding to the block according to the laser scanning range corresponding to the block for each block;

The scanning module 630 is configured to perform laser scanning on the block according to laser scanning parameters to obtain a 3D point cloud of the block.

The splicing module 640 is configured to splice the multiple segmented 3D point clouds to obtain the 3D point cloud of the region of interest.

Optionally, the blocking module 610 is specifically configured to analyze the scene scan image of the current scene to obtain the actual image quality parameter of the scene scan image; if the actual image quality parameter is less than the preset parameter threshold, then scan Extract regions of interest from images

Optionally, the blocking module 610 is also used to determine the laser scanning parameters corresponding to each laser scanning position according to the image quality model and target image quality parameters before extracting the region of interest from the scene scan image of the current scene, and the image quality The model is used to indicate the corresponding relationship between the laser scanning parameters and image quality parameters corresponding to each laser scanning position; at each laser scanning position, perform laser scanning on the current scene according to the corresponding laser scanning parameters to obtain the scene scanning image of the current scene .

Optionally, the blocking module 610 is specifically used to obtain the image quality model in the following manner: perform multiple laser scans on the sample scene to obtain multiple sample scan images, and different sample scan images correspond to different laser scan parameters; performing image analysis on multiple sample scan images to obtain the image quality parameters of each image position in the multiple sample scan images; using the image quality parameters of each image position in the multiple sample scan images, the laser scanning parameters corresponding to the multiple sample scan images, and The corresponding relationship between the predetermined image position and the laser scanning position is trained to obtain the image quality model.

Optionally, the blocking module 610 is specifically configured to, according to the image quality parameters of each image position in the multiple sample scan images and the corresponding relationship between the image position and the laser scan position, determine each laser scan image in the multiple sample scan images The image quality parameter corresponding to the position; the sample data is extracted from the laser scanning parameters corresponding to multiple sample scanning images and the image quality parameters corresponding to each laser scanning position in the multiple sample scanning images, the sample data includes laser scanning parameters, laser scanning position And the image quality parameters corresponding to the laser scanning parameters and the laser scanning position; the initial model is trained by using the sample data to obtain the image quality model.

Optionally, the block module 610 is specifically configured to determine the laser scanning range corresponding to the region of interest according to the setting parameters of the laser scanning device; obtain the block parameters, and divide the region of interest into multiple blocks according to the block parameters , and record the position information of each block in the region of interest; for each block, according to the position information of the block in the region of interest and the laser scanning range corresponding to the region of interest, determine the corresponding laser scanning range.

Optionally, the blocking module 610 is specifically used to obtain the regional image of the region of interest; analyze the abnormal points in the regional image to obtain the abnormal point data; use the pre-built evaluation model to analyze the abnormal point data and the target point cloud The quality parameters are processed to obtain the block parameters corresponding to the current scene, and the evaluation model is used to indicate the correspondence between abnormal point data, point cloud quality parameters and block parameters.

Optionally, the blocking module 610 is specifically configured, and the blocking parameters include: the number of blocks and the overlapping ratio.

Optionally, the block module 610 is specifically used for the abnormal point data including: the proportion of abnormal points; analyzing the abnormal point situation in the regional image to obtain the abnormal point data, including: identifying the abnormal points existing in the regional image, and calculating The proportion of abnormal points in the region image is obtained to obtain the proportion of abnormal points.

Optionally, the blocking module 610 is specifically configured to obtain the evaluation model in the following manner: acquire collected sample outlier data corresponding to multiple sample area images, sample block parameters corresponding to multiple sample area images, and Sample point cloud quality parameters corresponding to multiple sample area images; using sample abnormal point data corresponding to multiple sample area images, sample block parameters corresponding to multiple sample area images, and sample points corresponding to multiple sample area images Cloud quality parameters, construct a sample data set; perform training based on the sample data set, and construct an evaluation model.

Optionally, the blocking module 610 is specifically configured to acquire a plurality of sample area images, analyze the outliers in the multiple sample area images, and obtain sample outlier data corresponding to the multiple sample area images; for each The sample area of interest corresponding to the sample area image, block the sample area of interest according to the sample block parameters, obtain the block group corresponding to the sample block parameter, and determine the laser scanning range corresponding to each block in the block group ;According to the laser scanning range corresponding to each block, carry out laser scanning on each block to obtain the sample 3D point cloud of each block; analyze the sample 3D point cloud of multiple blocks to obtain the sample area The sample point cloud quality parameter corresponding to the image.

Optionally, the scanning module 630 is specifically configured to control the rotation of the vibrating mirror in the laser scanning device according to the laser scanning parameters, and use the laser reflected by the vibrating mirror to perform laser scanning on the block to obtain the 3D point cloud of the block.

Optionally, the splicing module 640 is specifically configured to, for the 3D point cloud of any two adjacent blocks, according to the position information of the two adjacent blocks in the region of interest, the 3D point cloud of the two adjacent blocks According to the point cloud quality of the overlapping area point cloud, the target overlapping area point cloud for splicing is selected from the overlapping area point cloud, and the target overlapping area point cloud is The cloud is spliced with the point cloud of the non-overlapping area; the 3D point cloud of the area of interest is obtained.

Optionally, the scanning module 630 further includes that the laser scanning parameters include any of the following: laser scanning angle range, laser signal intensity, or laser scanning speed.

According to the 3D point cloud processing device based on the dynamic frame provided in this embodiment, the region of interest in the scene scanning image is divided into multiple blocks, and part of the range is intercepted from the total range of laser scanning of the laser scanning device as the corresponding block. According to the laser scanning range corresponding to each block, configure the laser scanning parameters corresponding to each block, and perform laser scanning on the block according to the laser scanning parameters, so that the laser energy per unit time Concentration helps to obtain a better laser scanning effect and effectively improves the signal-to-noise ratio; by splicing multiple 3D point clouds of blocks, the 3D point cloud of the region of interest can be easily obtained, and effectively The accuracy of 3D point cloud is improved, the quality of point cloud is improved, and the processing method of point cloud is optimized. The present invention also provides a non-volatile computer storage medium. The computer storage medium stores at least one executable instruction, and the executable instruction can execute the dynamic frame-based 3D point cloud processing method in any method embodiment above.

FIG. 7 shows a schematic structural diagram of a computing device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in FIG. 7 , the computing device may include: a processor (processor) 702 , a communication interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

in:

The processor 702 , the communication interface 704 , and the memory 706 communicate with each other through the communication bus 708 .

The communication interface 704 is configured to communicate with network elements of other devices such as clients or other servers.

The processor 702 is configured to execute the program 710, specifically, may execute the relevant steps in the above embodiment of the dynamic frame-based 3D point cloud processing method.

Specifically, the program 710 may include program codes including computer operation instructions.

The processor 702 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the computing device may be of the same type, such as one or more CPUs, or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 706 is used for storing the program 710 . The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 710 may be specifically configured to enable the processor 702 to execute the dynamic frame-based 3D point cloud processing method in any of the above method embodiments. For the specific implementation of each step in the program 710, refer to the corresponding description of the corresponding steps and units in the above-mentioned embodiment of dynamic frame-based 3D point cloud processing, and details are not repeated here. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the foregoing method embodiments, and details are not repeated here.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings), as well as any method or method so disclosed, may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the embodiments of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

Claims (18)

1. A kind of 3D point cloud processing method based on dynamic frame, described method comprises:

   Extracting the region of interest from the scene scan image of the current scene, dividing the region of interest into a plurality of blocks, and determining the laser scanning range corresponding to each block;

   For each block, according to the laser scanning range corresponding to the block, obtain the laser scanning parameters corresponding to the block;

   Perform laser scanning on the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block;

   The multiple block 3D point clouds are spliced to obtain the 3D point cloud of the region of interest.
2. The method according to claim 1, wherein the extracting the region of interest from the scene scan image of the current scene comprises:

   Analyzing the scene scan image of the current scene to obtain the actual image quality parameters of the scene scan image;

   If the actual image quality parameter is smaller than the preset parameter threshold, the region of interest is extracted from the scene scan image of the current scene.
3. The method according to claim 1, wherein before extracting the region of interest from the scene scan image of the current scene, the method further comprises:

   According to the image quality model and the target image quality parameter, determine the laser scanning parameters corresponding to each laser scanning position, and the image quality model is used to indicate the corresponding relationship between the laser scanning parameters corresponding to each laser scanning position and the image quality parameter;

   At each laser scanning position, laser scanning is performed on the current scene according to corresponding laser scanning parameters to obtain a scene scanning image of the current scene.
4. The method according to claim 3, wherein the laser scanning parameters include any one of the following: laser signal intensity or laser scanning speed.
5. The method according to claim 3, wherein the image quality model is obtained as follows:

   Perform multiple laser scans on the sample scene to obtain multiple sample scan images, and different sample scan images correspond to different laser scan parameters;

   Image analysis is performed on the scanned images of multiple samples to obtain image quality parameters of each image position in the scanned images of multiple samples;

   The image quality model is obtained by using the image quality parameters of each image position in the multiple sample scan images, the laser scan parameters corresponding to the multiple sample scan images, and the predetermined correspondence between the image positions and the laser scan positions.
6. The method according to claim 5, wherein the image quality parameters of each image position in the multiple sample scan images, the laser scan parameters corresponding to the multiple sample scan images, and the predetermined image position and laser scan position are used The corresponding relationship between is trained to obtain an image quality model, including:

   Determining the image quality parameter corresponding to each laser scanning position in the multiple sample scanning images according to the image quality parameters of each image position in the multiple sample scanning images and the corresponding relationship between the image position and the laser scanning position;

   The sample data is extracted from the laser scanning parameters corresponding to the multiple sample scanning images and the image quality parameters corresponding to the laser scanning positions in the multiple sample scanning images, the sample data includes the laser scanning parameters, the laser scanning position and the Scanning parameters and image quality parameters corresponding to the laser scanning position;

   The initial model is trained by using the sample data to obtain an image quality model.
7. The method according to claim 1, wherein said dividing said region of interest into a plurality of sub-blocks, and determining the laser scanning range corresponding to each sub-block comprises:

   Determine the laser scanning range corresponding to the region of interest according to the setting parameters of the laser scanning device;

   Obtaining block parameters, dividing the region of interest into multiple blocks according to the block parameters, and recording the position information of each block in the region of interest;

   For each block, the laser scanning range corresponding to each block is determined according to the position information of the block in the region of interest and the laser scanning range corresponding to the region of interest.
8. The method according to claim 7, wherein said obtaining block parameters comprises:

   Obtain an area image of the area of interest;

   Analyzing abnormal points in the image of the region to obtain abnormal point data;

   Process the abnormal point data and target point cloud quality parameters with a pre-built evaluation model to obtain block parameters corresponding to the current scene, and the evaluation model is used to indicate the abnormal point data, point cloud quality parameters and block parameters corresponding relationship.
9. The method according to claim 8, wherein the block parameters include: the number of blocks and the overlap rate.
10. The method according to claim 8, wherein the abnormal point data includes: the proportion of abnormal points;

    The analysis of the abnormal points in the image of the region is carried out to obtain the abnormal point data, including:

    Identifying the abnormal points existing in the region image, calculating the ratio of the abnormal points in the region image, and obtaining the ratio of the abnormal points.
11. The method according to claim 8, wherein the evaluation model is obtained in the following manner:

    Obtaining collected sample outlier data corresponding to multiple sample area images, sample block parameters corresponding to multiple sample area images, and sample point cloud quality parameters corresponding to multiple sample area images;

    Constructing a sample data set by using the sample outlier data corresponding to the multiple sample area images, the sample block parameters corresponding to the multiple sample area images, and the sample point cloud quality parameters corresponding to the multiple sample area images;

    Training is carried out according to the sample data set, and an evaluation model is constructed.
12. The method according to claim 11, characterized in that the acquiring collected sample outlier data corresponding to a plurality of sample area images, sample block parameters corresponding to a plurality of sample area images, and data corresponding to a plurality of sample area images The sample point cloud quality parameters corresponding to the image, including:

    Obtaining a plurality of sample area images, analyzing abnormal points in the plurality of sample area images, and obtaining sample abnormal point data corresponding to the plurality of sample area images;

    For the sample region of interest corresponding to each sample region image, block the sample region of interest according to the sample block parameter, obtain the block group corresponding to the sample block parameter, and determine the block group The laser scanning range corresponding to each block;

    According to the laser scanning range corresponding to each block, laser scanning is performed on each block to obtain the sample 3D point cloud of each block;

    The sample 3D point cloud of multiple blocks is analyzed to obtain the sample point cloud quality parameter corresponding to the image of the sample area.
13. The method according to any one of claims 1 to 12, wherein the laser scanning of the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block includes:

    According to the laser scanning parameters, the rotation of the vibrating mirror in the laser scanning device is controlled, and the laser beam reflected by the vibrating mirror is used to perform laser scanning on the block to obtain the 3D point cloud of the block.
14. The method according to any one of claims 1 to 12, wherein the splicing of multiple segmented 3D point clouds to obtain the 3D point cloud of the region of interest comprises:

    For the 3D point clouds of any two adjacent blocks, according to the position information of the two adjacent blocks in the region of interest, perform intersection processing on the 3D point clouds of the two adjacent blocks , get point cloud of overlapping area and point cloud of non-overlapping area;

    According to the point cloud quality of the overlapping area point cloud, select the target overlapping area point cloud for splicing from the overlapping area point cloud, and perform splicing processing on the target overlapping area point cloud and the non-overlapping area point cloud. ;

    A 3D point cloud of the region of interest is obtained.
15. The method according to any one of claims 1 to 12, wherein the laser scanning parameters include any of the following: laser scanning angle range, laser signal intensity or laser scanning speed.
16. A 3D point cloud processing device based on a dynamic frame, for performing the method according to any one of claims 1 to 15, said device comprising:

    A block module, configured to extract the region of interest from the scene scan image of the current scene, divide the region of interest into multiple blocks, and determine the laser scanning range corresponding to each block;

    The obtaining module is used for obtaining the laser scanning parameters corresponding to the block according to the laser scanning range corresponding to the block for each block;

    The scanning module is used to perform laser scanning on the block according to the laser scanning parameters corresponding to the block to obtain the 3D point cloud of the block;

    The splicing module is configured to splice the multiple segmented 3D point clouds to obtain the 3D point cloud of the region of interest.
17. A computing device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete mutual communication through the communication bus;

    The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the dynamic frame-based 3D point cloud processing method according to any one of claims 1 to 15.
18. A computer storage medium, at least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to execute the dynamic frame-based 3D point cloud processing method according to any one of claims 1 to 15 corresponding operation.