CN115861962A - Point cloud filtering method and device, electronic equipment and storage medium - Google Patents
Point cloud filtering method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115861962A CN115861962A CN202211426309.8A CN202211426309A CN115861962A CN 115861962 A CN115861962 A CN 115861962A CN 202211426309 A CN202211426309 A CN 202211426309A CN 115861962 A CN115861962 A CN 115861962A
- Authority
- CN
- China
- Prior art keywords
- feature
- point cloud
- image
- concentration
- current frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention provides a point cloud filtering method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring horizontal images acquired by a plurality of vision sensors with horizontal visual angles; converting each horizontal image into a top view image of a top view angle; splicing the plurality of overlooking images into a ring-view image according to the visual angle of each visual sensor; calculating a density probability map of the current frame of all-around image according to at least one frame of all-around image before the current frame, wherein the feature density of the corresponding current frame of all-around image in the density probability map is used for indicating whether the corresponding feature is a dynamic barrier; and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability map. The invention realizes the filtering of dynamic obstacles in the point cloud data on the premise of not needing a large amount of laser point cloud data.
Description
Technical Field
The invention relates to the field of machine vision and image processing, in particular to a point cloud filtering method and device, electronic equipment and a storage medium.
Background
At present, in the field of automatic driving, map building and vehicle positioning are often required to be carried out by means of laser point clouds collected by laser sensors. The processing of dynamic point clouds is particularly important in map building and vehicle localization based on laser point clouds. The existing dynamic processing can only identify whether the laser point cloud corresponds to a dynamic obstacle according to the morphological structure of the laser point cloud, however, a large amount of laser point cloud data is needed to distinguish different object types.
Therefore, how to filter out dynamic obstacles in point cloud data without a large amount of laser point cloud data is a technical problem to be solved urgently by technical personnel in the field.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a point cloud filtering method, a point cloud filtering device, electronic equipment and a storage medium, so that dynamic barrier filtering in point cloud data is realized on the premise of not needing a large amount of laser point cloud data.
According to an aspect of the present invention, there is provided a point cloud filtering method, including:
acquiring horizontal images acquired by a plurality of vision sensors with horizontal visual angles;
converting each horizontal image into a top-view image of a top-view visual angle;
splicing the plurality of overlooking images into a ring-view image according to the visual angle of each visual sensor;
calculating a density probability map of the current frame of all-around image according to at least one frame of all-around image before the current frame, wherein the feature density of the corresponding current frame of all-around image in the density probability map is used for indicating whether the corresponding feature is a dynamic barrier;
and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability map.
In some embodiments of the present application, the converting each of the horizontal images into a top view image of a top view perspective includes:
and segmenting the overlook image to obtain segmentation features and feature labels.
In some embodiments of the present application, the filtering, based on the concentration probability map, the laser point cloud at the feature position where the feature concentration conforms to the dynamic obstacle concentration range includes:
and assigning the feature labels of the segmentation features to the filtered and retained laser point cloud.
In some embodiments of the present application, the feature density of the surrounding image corresponding to the current frame in the density probability map is calculated according to the relative motion distance difference between the corresponding features of the adjacent frames, the distance error, and the feature density of the corresponding feature of the previous frame.
In some embodiments of the present application, the calculating a density probability map of the current frame all-round looking image according to at least one frame all-round looking image before the current frame includes:
aligning each feature in the previous frame of all-around image to the current time according to the acquisition time;
calculating the relative motion distance difference and distance error between each feature in the previous frame of the aligned panoramic image and the corresponding feature in the current frame of the panoramic image;
and calculating the density probability chart of the current frame of the surround-view image according to the relative motion distance difference and the distance error between the corresponding features of the previous frame of the surround-view image and the current frame of the surround-view image and the feature density of the corresponding features of the previous frame of the surround-view image.
In some embodiments of the present application, the concentration probability map includes a concentration of a feature corresponding to the panoramic image of the current frame
Calculated according to the following formula:
wherein Q is dis Is the relative motion distance difference, Q, between corresponding features label of the previous and current frames of the panoramic image miss In order to be a distance error,
the feature density of the corresponding feature label of the previous frame of the panoramic image.
In some embodiments of the present application, the filtering the laser point cloud at the feature having the feature concentration conforming to the dynamic obstacle concentration range based on the concentration probability map comprises:
and converting the current frame panoramic image into a laser coordinate system, and filtering laser point clouds at the characteristic position of which the characteristic concentration accords with the concentration range of the dynamic obstacle based on the concentration probability map.
According to still another aspect of the present application, there is also provided a point cloud filtering apparatus including:
the horizontal image acquisition module is used for acquiring horizontal images acquired by a plurality of vision sensors with horizontal visual angles;
the overlook visual angle conversion module is used for converting each horizontal image into an overlook image of an overlook visual angle;
the all-round stitching module is used for stitching the plurality of overlooking images into all-round images according to the visual angle of each visual sensor;
the concentration probability map calculation module is used for calculating a concentration probability map of the current frame of all-around-looking image according to at least one frame of all-around-looking image before the current frame, and the feature concentration of the all-around-looking image corresponding to the current frame in the concentration probability map is used for indicating whether the corresponding feature is a dynamic barrier or not;
and the laser point cloud filtering module is used for filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability graph.
According to still another aspect of the present invention, there is also provided an electronic apparatus, including: a processor; a storage medium having stored thereon a computer program which, when executed by the processor, performs the steps as described above.
According to yet another aspect of the present invention, there is also provided a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps as described above.
Compared with the prior art, the invention has the advantages that:
the horizontal images acquired by the vision sensor are converted into overlook images and then are spliced, so that a concentration probability graph corresponding to the spliced overlook images is calculated, the probability that each feature is a dynamic obstacle is obtained, and the laser point cloud at the feature position with the feature concentration conforming to the concentration range of the dynamic obstacle can be filtered based on the calculated concentration probability graph, so that the dynamic obstacle of the laser point cloud is filtered. Therefore, the filtering of the laser point clouds can be realized without carrying out object classification on a large number of laser point clouds, the concentration probability map carries out the laser point cloud filtering on the basis of the probability, the frame difference result of the current frame is not completely trusted, certain anti-interference performance is achieved on the subsequent semantic segmentation effect such as a top view, and therefore the filtered laser point clouds can obtain higher accuracy when carrying out vehicle positioning and map construction. The method and the device are particularly suitable for scenes with more dynamic obstacles, so that the dynamic obstacles can be prevented from interfering with the positioning of the vehicle.
Drawings
The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
FIG. 1 shows a flow diagram of a point cloud filtering method according to an embodiment of the invention;
FIG. 2 is a schematic diagram illustrating a top view image of a top view perspective obtained using transform model transformation, according to an embodiment of the invention;
FIG. 3 illustrates a schematic diagram of a top view image of a top view perspective obtained using homographic matrix translation, according to an embodiment of the invention;
FIG. 4 illustrates a schematic view of alignment of a look-around image by acquisition time according to an embodiment of the present invention;
FIG. 5 illustrates a flow diagram of a point cloud filtering method in accordance with a specific embodiment of the present invention;
FIG. 6 illustrates a block diagram of a point cloud filtering apparatus according to an embodiment of the invention;
FIG. 7 schematically illustrates a computer-readable storage medium in an exemplary embodiment of the disclosure;
fig. 8 schematically illustrates an electronic device in an exemplary embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In order to solve the defects of the prior art, the invention provides a point cloud filtering method. Referring now to FIG. 1, FIG. 1 shows a flow diagram of a point cloud filtering method according to an embodiment of the invention. FIG. 1 comprises the following steps:
step S110: acquiring horizontal images acquired by a plurality of vision sensors with the visual angles in the horizontal direction.
Step S120: and converting each horizontal image into a top-view image of a top-view viewing angle.
Step S130: and splicing the plurality of overlooking images into a ring-view image according to the visual angle of each visual sensor.
Step S140: and calculating a density probability map of the current frame of all-around image according to at least one frame of all-around image before the current frame, wherein the feature density of the corresponding current frame of all-around image in the density probability map is used for indicating whether the corresponding feature is a dynamic obstacle or not.
Step S150: and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability map.
In the point cloud filtering method provided by the invention, the horizontal images acquired by the vision sensor are converted into the overlook images and then are spliced, so that the concentration probability graph corresponding to the spliced overlook images is calculated, the probability that each feature is a dynamic obstacle is obtained, and the laser point cloud at the feature position with the feature concentration conforming to the concentration range of the dynamic obstacle can be filtered based on the calculated concentration probability graph, so that the dynamic obstacle of the laser point cloud is filtered. Therefore, the filtering of the laser point clouds can be realized without carrying out object classification on a large number of laser point clouds, the concentration probability map carries out the laser point cloud filtering on the basis of the probability, the frame difference result of the current frame is not completely trusted, certain anti-interference performance is achieved on the subsequent semantic segmentation effect such as a top view, and therefore the filtered laser point clouds can obtain higher accuracy when carrying out vehicle positioning and map construction. The method and the device are particularly suitable for scenes with more dynamic obstacles, so that the dynamic obstacles can be prevented from interfering with the positioning of the vehicle.
Specifically, the vision sensor may be, for example, a camera module, and the viewing angle in the horizontal direction may have a certain deviation amount, that is, a deviation angle in the vertical direction may be based on the horizontal direction. Further, images collected by the vision sensor with a plurality of visual angles in the horizontal direction can cover the 360-degree range of the vehicle body, so that the subsequent all-round-looking splicing step is facilitated. In some embodiments, a vision sensor may be provided in each of four directions of the vehicle body. For example, the vehicle may be provided at four corners of the vehicle body or at the midpoint of four surfaces of the vehicle, and the present application is not limited thereto, and the position at which each of the vision sensors is provided is not limited thereto. Meanwhile, the setting position and the setting parameter of each vision sensor can be stored at the vehicle end and/or the server end so as to determine the visual angle range of each vision sensor in the subsequent all-round stitching step.
Specifically, step S120 may convert the horizontal image of the horizontal viewing angle into the top view image of the top view angle through the Transformer model. The perspective transformation of the transform model does not generate distortion compared to the transformation method using the homography matrix, as shown in fig. 2 and 3. Further, in step S120, the overhead view image may be segmented to obtain segmentation features and feature labels. In particular, image segmentation may be implemented based on a deep learning model. The deep learning model can distinguish pixel regions of different objects in the image. Deep learning models may include, but are not limited to, VGG16, FCN (full convolution network), segNet (split network), deep lab, etc., and the application is not so limited. After the planar image is divided to obtain the division features, a feature label may be assigned to each division feature. Learning may be based on image samples of a plurality of different objects, so that feature labels of respective feature regions (segmentation features) can be recognized during or after segmentation. Further, since the number of image samples of different objects is large, it is easier to train than laser point cloud so as to identify the object type of each feature region (segmentation feature), and to assign a corresponding feature label.
In some implementations, after the top-view image is segmented to obtain the segmentation features and the feature labels in step S120, a preliminary screening of dynamic obstacles may also be performed based on the feature labels. For example, the feature labels are segmented features of pedestrians, vehicles and the like, which are necessarily dynamic obstacles, so that the marking can be performed in advance, the subsequent calculation of feature concentration is not needed, and the filtering can be performed in the laser point cloud. Specifically, in this embodiment, the feature labels can be subjected to preliminary screening to conform the feature labels to the segmentation features of the preset dynamic obstacle feature labels, and filtering is directly performed at the positions of the corresponding segmentation features of the laser point cloud; and keeping the segmentation features of which the feature labels do not accord with the preset dynamic barrier feature labels, and executing the calculation of a concentration probability graph and feature concentration in the subsequent steps so as to judge whether the segmentation features are dynamic barriers or not through the calculation of the feature concentration. Further, in some variations, static feature labels, such as plants, buildings, and the like, may also be maintained, and since these static features are inevitably not moved, the feature labels of the segmented features are also determined and then preliminarily screened, so that the computation of the concentration probability map and the feature concentration in the subsequent steps is not required to be performed on the segmented features whose feature labels conform to the static feature labels, and the laser point clouds corresponding to the segmented features that conform to the static feature labels will be retained without filtering. Therefore, the calculation amount of the characteristic concentration is reduced, the laser point cloud filtering efficiency is improved, and the system load is reduced.
In some implementations, after the top-view image is segmented to obtain the segmented features and the feature labels in step S120, the feature labels of the segmented features can be assigned to the filtered retained laser point cloud after step S150. Therefore, object identification and feature labeling of the laser point cloud are not needed, the calculation amount of giving the feature labels to the laser point cloud is reduced, the object identification efficiency of the laser point cloud is improved, and the system load is reduced.
Specifically, the density probability map includes each feature (segmentation feature) in the current frame panoramic image and its feature density. The feature density of the surround-view image of the current frame in the density probability map can be calculated according to the relative motion distance difference between the corresponding features of the adjacent frames, the distance error and the feature density of the corresponding features of the previous frame. The relative motion distance difference can be used to indicate whether the corresponding feature of the adjacent frame is in a motion state. The distance error may be set to select an allowable distance error, and an error due to the deep learning process may be reduced. The feature concentration of the corresponding feature of the previous frame is used to provide a reference of whether the corresponding feature was in motion in the previous frame. Therefore, the feature density of the surrounding image corresponding to the current frame in the density probability map is calculated according to the relative motion distance difference between the corresponding features of the adjacent frames, the distance error and the feature density of the corresponding features of the previous frame, so that the calculated feature density can be used for indicating whether the corresponding features are dynamic obstacles or not.
Specifically, the calculating the density probability map of the current frame all-around view image according to at least one frame all-around view image before the current frame may include the following steps: aligning each feature in the previous frame of all-round-looking image to the current time according to the acquisition time; calculating the relative motion distance difference and distance error between each feature in the previous frame of the registered panoramic image and the corresponding feature in the current frame of the panoramic image; and calculating the density probability chart of the current frame of the surround-view image according to the relative motion distance difference and the distance error between the corresponding features of the previous frame of the surround-view image and the current frame of the surround-view image and the feature density of the corresponding features of the previous frame of the surround-view image.
Specifically, the density probability map has a feature density corresponding to the panoramic image of the current frame
Can be calculated according to the following formula:
wherein Q dis Is the relative motion distance difference, Q, between corresponding features label of the previous and current frames of the panoramic image miss In order to be a distance error,
the feature density of the corresponding feature label of the previous frame of the panoramic image. Therefore, the method and the device can acquire the multiple frames of all-around images before the current moment so as to sequentially calculate and obtain the motion state of the feature in the continuous time from the first frame of image through the formula.
Further, an initial value may be set for the feature density of the first frame image, and the initial value may be set as needed, which is not limited in this application. For example, at setting up
Can be based on>
Calculating the relative motion distance difference and the distance error of the feature label in the first frame image and the second frame image to obtain ^ 4>
Then according to>
The relative motion distance difference and the distance error of the feature label in the second frame image and the third frame image are calculated to obtain ^ 4>
And the like until the current frame is calculated.
Further, when calculating the feature density of the corresponding current frame all-round looking image in the density probability map, the number of multi-frame all-round looking images before the current time to be acquired may be determined, so as to balance the calculation speed of the feature density of the corresponding current frame all-round looking image in the density probability map, and determine whether the continuous time of whether the corresponding feature is in the motion state is long enough. In some embodiments, the number of frames of the surround view image before the current time to be acquired may be manually set. In other embodiments, different numbers of the multi-frame all-round images before the current time to be acquired may be tested to determine whether the concentration of the feature in the concentration probability map obtained by different numbers and corresponding to the current frame all-round image can accurately indicate whether the feature is a moving obstacle, so as to determine the number of the multi-frame all-round images before the current time to be acquired according to the calculation accuracy. In still other embodiments, the number of multiple frames of surround view images before the current time to be acquired may also be predicted based on an artificial intelligence model. The present application can implement more variations, which are not described herein.
Further, the distance error may be set as desired. In some variations, the distance error may also be obtained by calculating a mode of the error pixel, and this application may implement more variation modes, which are not described herein again.
Further, the dynamic barrier concentration range may be set as desired. For example, the dynamic barrier concentration range may be less than 1. The application is not so limited and the dynamic barrier concentration range may be less than 2, 3, etc. In some specific implementations, the dynamic obstacle concentration range may be set in synchronization with the number of the multiple frames of panoramic images to be acquired before the current time, and the dynamic obstacle concentration range may be determined according to different feature concentrations because the feature concentrations of the concentration probability map output by different numbers of the multiple frames of panoramic images to be acquired before the current time, which correspond to the current frame of panoramic images, are different. The present application can implement more variations, which are not described herein.
Specifically, the relative movement distance (X) of the image captured by the vision sensor at each time can be determined according to the travel of the vehicle 1 ，X 2 ，......X t ) Wherein X is 1 ，X 2 ，......X t Which respectively represent the distance of the relative movement between the 1 st and 2 nd. Thus, the relative movement distance difference between each feature in the previous frame of the all-around image and the corresponding feature in the current frame of the all-around image can be calculated based on the aligned all-around images. For example, when a feature is not moving, the position of each frame is unchanged in the aligned frames of the panoramic image, so that the obtained relative movement distance difference is 0; when a feature is not moving, the position of each frame is changed after alignment, and therefore the distance between the same features in adjacent frames can be used as the relative movement distance difference. If the alignment processing is not carried out, the vehicle can cause the feature which is not moved to generate a distance difference between frames due to relative vehicle movement when the vehicle moves, and the distance difference is not caused by the motion of the featureTherefore, it is difficult to obtain a distance difference caused by the movement of the feature itself based directly on the distance calculation of the corresponding feature in the image, and the above-described looking-around images at each time are aligned, that is, the influence of the vehicle travel on the feature is removed, so that an accurate relative movement distance difference can be obtained. In some variations, the present application may also eliminate the relative movement distance caused by the influence of vehicle travel when calculating the relative motion distance difference of the corresponding features in the adjacent frames without performing alignment. The present application can implement more variations, which are not described herein.
Specifically, step S150 may further include the steps of: and converting the current frame looking-around image into a laser coordinate system, and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the concentration range of the dynamic barrier based on the concentration probability graph. Thus, the panoramic image and the laser point cloud can be in the same coordinate dimension to facilitate determination of the corresponding features.
Referring now to FIG. 5, FIG. 5 illustrates a flow chart of a point cloud filtering method according to an embodiment of the invention. Fig. 5 shows the following steps in total:
step S310: acquiring horizontal images acquired by a plurality of vision sensors with the visual angles in the horizontal direction.
Step S320: and converting each horizontal image into a top view image of a top view angle, and segmenting the top view image to obtain segmentation features and feature labels.
Step S330: and splicing the plurality of overlooking images into a surround view image according to the visual angle of each visual sensor.
Step S340: and calculating a density probability map of the current frame of all-around image according to at least one frame of all-around image before the current frame, wherein the feature density of the corresponding current frame of all-around image in the density probability map is used for indicating whether the corresponding feature is a dynamic obstacle or not.
Step S350: and converting the current frame panoramic image into a laser coordinate system.
Step S360: and acquiring laser point cloud collected by a laser sensor. Wherein the laser sensor and the vision sensor are mounted on the same vehicle.
Step S370: and clustering and dividing the laser point cloud to obtain a plurality of point cloud characteristics so as to correspond to the characteristics of the all-round looking image.
Step S380: and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability map.
Step S390: and assigning the feature labels of the segmentation features to the filtered and retained laser point cloud.
Specifically, the laser point cloud is clustered in step S360 to obtain a plurality of point cloud features, and after filtering in step S380, part of the point cloud features are retained, and the point cloud features and the segmentation features have corresponding relations in the overlooking position. In some embodiments, since both the point cloud feature and the segmentation feature have the region range, the corresponding relationship between the point cloud feature and the segmentation feature may be determined according to the overlapping degree of the region range, for example, an overlapping degree threshold may be set, and when the overlapping degree is greater than the set overlapping degree threshold, the corresponding relationship between the point cloud feature and the segmentation feature may be determined, so that the feature label of the segmentation feature may be assigned to the filtered and retained laser point cloud. In other embodiments, since both the point cloud feature and the segmentation feature have the region range, the corresponding relationship between the point cloud feature and the segmentation feature may also be determined according to the distance between the region centers, for example, a distance threshold may be set, and when the distance between the point cloud feature and the region center of the segmentation feature is smaller than the set distance threshold, the corresponding relationship between the point cloud feature and the segmentation feature may be determined, so that the feature label of the segmentation feature may be assigned to the filtered and retained laser point cloud. According to the method and the device, more change modes for determining the corresponding relation between the point cloud characteristics and the segmentation characteristics can be realized, and details are not repeated herein.
The above are only a plurality of specific implementation manners of the point cloud filtering method of the present invention, and each implementation manner may be implemented independently or in combination, and the present invention is not limited thereto. Furthermore, the flow charts of the present invention are merely schematic, the execution sequence between the steps is not limited thereto, and the steps can be split, combined, exchanged sequentially, or executed synchronously or asynchronously in other ways within the protection scope of the present invention.
The invention also provides a point cloud filtering device, and fig. 6 shows a block diagram of the point cloud filtering device according to the embodiment of the invention. The point cloud filtering apparatus 400 includes a horizontal image obtaining module 410, a top view angle converting module 420, a look-around stitching module 430, a density probability map calculating module 440, and a laser point cloud filtering module 450.
The horizontal image acquiring module 410 is configured to acquire horizontal images acquired by a plurality of vision sensors with horizontal viewing angles;
the top view conversion module 420 is configured to convert each of the horizontal images into a top view image of a top view;
the around-looking stitching module 430 is configured to stitch the multiple top-view images into an around-looking image according to the viewing angle of each of the vision sensors;
the density probability map calculation module 440 is configured to calculate a density probability map of the current frame of surround-view image according to at least one frame of surround-view image before the current frame, where a feature density of the current frame of surround-view image in the density probability map is used to indicate whether a corresponding feature is a dynamic obstacle;
the laser point cloud filtering module 450 is configured to filter the laser point cloud at the feature position where the feature concentration conforms to the dynamic obstacle concentration range based on the concentration probability map.
In the point cloud filtering device provided by the invention, the horizontal images acquired by the vision sensor are converted into the overlook images and then are spliced, so that the concentration probability graph corresponding to the spliced overlook images is calculated, the probability that each feature is a dynamic obstacle is obtained, and the laser point cloud at the feature position with the feature concentration conforming to the concentration range of the dynamic obstacle can be filtered based on the calculated concentration probability graph, so that the dynamic obstacle of the laser point cloud is filtered. Therefore, the filtering of the laser point clouds can be realized without carrying out object classification on a large number of laser point clouds, the concentration probability map carries out the laser point cloud filtering on the basis of the probability, the frame difference result of the current frame is not completely trusted, certain anti-interference performance is achieved on the subsequent semantic segmentation effect such as a top view, and therefore the filtered laser point clouds can obtain higher accuracy when carrying out vehicle positioning and map construction. The method and the device are particularly suitable for scenes with more dynamic obstacles, so that the dynamic obstacles can be prevented from interfering with the positioning of the vehicle.
Fig. 6 is a schematic diagram illustrating a point cloud filtering apparatus 400 provided by the present invention, respectively, and the splitting, combining, and adding of modules are within the scope of the present invention without departing from the spirit of the present invention. The point cloud filtering apparatus 400 provided by the present invention can be implemented by software, hardware, firmware, plug-in, and any combination thereof, which is not limited thereto.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which when executed by, for example, a processor, may implement the steps of the point cloud filtering method described in any one of the above embodiments. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the invention described in the point cloud filtering method section above of this specification when the program product is run on the terminal device.
Referring to fig. 7, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the tenant computing device, partly on the tenant device, as a stand-alone software package, partly on the tenant computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing devices may be connected to the tenant computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In an exemplary embodiment of the present disclosure, there is also provided an electronic device, which may include a processor, and a memory for storing executable instructions of the processor. Wherein the processor is configured to perform the steps of the point cloud filtering method of any of the above embodiments via execution of the executable instructions.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 600 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the electronic device 600 is in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 that connects the various system components (including the storage unit 620 and the processing unit 610), a display unit 640, and the like.
Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention described in the point cloud filtering method section above in this specification. For example, the processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a tenant to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, to name a few.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the point cloud filtering method according to the embodiments of the present disclosure.
Compared with the prior art, the invention has the advantages that:
the horizontal images acquired by the vision sensor are converted into overlook images and then are spliced, so that a concentration probability graph corresponding to the spliced overlook images is calculated, the probability that each feature is a dynamic obstacle is obtained, and the laser point cloud at the feature position with the feature concentration conforming to the concentration range of the dynamic obstacle can be filtered based on the calculated concentration probability graph, so that the dynamic obstacle of the laser point cloud is filtered. Therefore, the filtering of the laser point clouds can be realized without carrying out object classification on a large number of laser point clouds, the concentration probability map carries out the laser point cloud filtering on the basis of the probability, the frame difference result of the current frame is not completely trusted, certain anti-interference performance is achieved on the subsequent semantic segmentation effect such as a top view, and therefore the filtered laser point clouds can obtain higher accuracy when carrying out vehicle positioning and map construction. The method and the device are particularly suitable for scenes with more dynamic obstacles, so that the dynamic obstacles can be prevented from interfering with the positioning of the vehicle.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (10)
1. A point cloud filtering method is characterized by comprising the following steps:
acquiring horizontal images acquired by a plurality of vision sensors with horizontal visual angles;
converting each horizontal image into a top view image of a top view angle;
splicing the plurality of overlooking images into a ring-view image according to the visual angle of each visual sensor;
calculating a density probability map of the current frame of all-around image according to at least one frame of all-around image before the current frame, wherein the feature density of the corresponding current frame of all-around image in the density probability map is used for indicating whether the corresponding feature is a dynamic barrier;
and filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability map.
2. The point cloud filtering method of claim 1, wherein said converting each of said horizontal images into a top view image of a top view perspective comprises:
and segmenting the overlook image to obtain segmentation features and feature labels.
3. The point cloud filtering method of claim 2, wherein a feature concentration of the concentration probability map corresponding to the current frame around-view image is used to indicate a probability that the corresponding segmented feature is a dynamic obstacle, and wherein the filtering the laser point cloud at the feature having the feature concentration corresponding to the dynamic obstacle concentration range based on the concentration probability map comprises:
and assigning the feature labels of the segmentation features to the filtered and retained laser point cloud.
4. The point cloud filtering method of claim 1, wherein the feature density of the surrounding image of the current frame in the density probability map is calculated according to the relative motion distance difference between the corresponding features of the adjacent frames, the distance error, and the feature density of the corresponding feature of the previous frame.
5. The point cloud filtering method of claim 4, wherein said calculating a density probability map of the current frame surround view image from at least one frame of surround view image preceding the current frame comprises:
aligning each feature in the previous frame of all-around image to the current time according to the acquisition time;
calculating the relative motion distance difference and distance error between each feature in the previous frame of the aligned panoramic image and the corresponding feature in the current frame of the panoramic image;
and calculating a density probability chart of the current frame of the looking-around image according to the relative motion distance difference and the distance error between the corresponding features of the previous frame of the looking-around image and the current frame of the looking-around image and the feature density of the corresponding features of the previous frame of the looking-around image.
6. The point cloud filtering method of claim 5, wherein the concentration probability map has a concentration of features corresponding to the current frame of the panoramic image
Calculated according to the following formula:
wherein Q dis Is the relative motion distance difference, Q, between corresponding features label of the previous and current frames of the panoramic image miss In order to be a distance error,
the feature density of the corresponding feature label of the previous frame of the panoramic image.
7. The point cloud filtering method of claim 1, wherein said filtering the laser point cloud at features whose feature concentrations conform to a dynamic obstacle concentration range based on the concentration probability map comprises:
and converting the current frame panoramic image into a laser coordinate system, and filtering laser point clouds at the characteristic position of which the characteristic concentration accords with the concentration range of the dynamic obstacle based on the concentration probability map.
8. A point cloud filtering device, comprising:
the horizontal image acquisition module is used for acquiring horizontal images acquired by a plurality of vision sensors with horizontal visual angles;
the overlook visual angle conversion module is used for converting each horizontal image into an overlook image of an overlook visual angle;
the all-round stitching module is used for stitching the plurality of overlooking images into all-round images according to the visual angle of each visual sensor;
the concentration probability map calculation module is used for calculating a concentration probability map of the current frame of all-around-looking image according to at least one frame of all-around-looking image before the current frame, and the feature concentration of the all-around-looking image corresponding to the current frame in the concentration probability map is used for indicating whether the corresponding feature is a dynamic barrier or not;
and the laser point cloud filtering module is used for filtering the laser point cloud at the characteristic position of which the characteristic concentration accords with the dynamic obstacle concentration range on the basis of the concentration probability graph.
9. An electronic device, characterized in that the electronic device comprises:
a processor;
storage medium having stored thereon a computer program which, when being executed by the processor, carries out the point cloud filtering method of any one of claims 1 to 7.
10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, performs the point cloud filtering method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211426309.8A CN115861962A (en) | 2022-11-15 | 2022-11-15 | Point cloud filtering method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211426309.8A CN115861962A (en) | 2022-11-15 | 2022-11-15 | Point cloud filtering method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115861962A true CN115861962A (en) | 2023-03-28 |
Family
ID=85663519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211426309.8A Pending CN115861962A (en) | 2022-11-15 | 2022-11-15 | Point cloud filtering method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861962A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758518A (en) * | 2023-08-22 | 2023-09-15 | 安徽蔚来智驾科技有限公司 | Environment sensing method, computer device, computer-readable storage medium and vehicle |
-
2022
- 2022-11-15 CN CN202211426309.8A patent/CN115861962A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758518A (en) * | 2023-08-22 | 2023-09-15 | 安徽蔚来智驾科技有限公司 | Environment sensing method, computer device, computer-readable storage medium and vehicle |
| CN116758518B (en) * | 2023-08-22 | 2023-12-01 | 安徽蔚来智驾科技有限公司 | Environment sensing method, computer device, computer-readable storage medium and vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111402414A (en) | Point cloud map construction method, device, equipment and storage medium | |
| CN112001912B (en) | Target detection method and device, computer system and readable storage medium | |
| CN112258519B (en) | Automatic extraction method and device for way-giving line of road in high-precision map making | |
| CN113281780B (en) | Method and device for marking image data and electronic equipment | |
| CN111383204A (en) | Video image fusion method, fusion device, panoramic monitoring system and storage medium | |
| CN113393448A (en) | Deformation detection method, device and equipment and computer readable storage medium | |
| CN111382695A (en) | Method and apparatus for detecting boundary points of object | |
| CN115050005B (en) | Target detection method and detection device for high-level video intelligent parking scene | |
| CN111242847B (en) | Gateway-based image splicing method, system, equipment and storage medium | |
| CN115861962A (en) | Point cloud filtering method and device, electronic equipment and storage medium | |
| CN109300322B (en) | Guideline drawing method, apparatus, device, and medium | |
| CN110111018B (en) | Method, device, electronic equipment and storage medium for evaluating vehicle sensing capability | |
| CN114972758A (en) | Instance segmentation method based on point cloud weak supervision | |
| CN114565906A (en) | Obstacle detection method, obstacle detection device, electronic device, and storage medium | |
| CN112489240B (en) | Commodity display inspection method, inspection robot and storage medium | |
| CN114140612A (en) | Method, device, equipment and storage medium for detecting hidden danger of power equipment | |
| CN111951328A (en) | Object position detection method, device, equipment and storage medium | |
| CN113239931A (en) | Logistics station license plate recognition method | |
| CN109934185B (en) | Data processing method and device, medium and computing equipment | |
| CN110853098A (en) | Robot positioning method, device, equipment and storage medium | |
| CN115994917A (en) | Automatic box grabbing method and device for unmanned straddle carrier, electronic equipment and storage medium | |
| CN112434674B (en) | Method, system and equipment for detecting tiny human face based on virtual amplification of label box | |
| CN114972945A (en) | Multi-machine-position information fusion vehicle identification method, system, equipment and storage medium | |
| CN111950420A (en) | Obstacle avoidance method, device, equipment and storage medium | |
| CN115731104A (en) | Vehicle vision sensor-based all-round-looking splicing method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 503-3, 398 Jiangsu Road, Changning District, Shanghai 200050 Applicant after: Shanghai Xijing Technology Co.,Ltd. Address before: Room 503-3, 398 Jiangsu Road, Changning District, Shanghai 200050 Applicant before: SHANGHAI WESTWELL INFORMATION AND TECHNOLOGY Co.,Ltd. |