CN114066994A - Point cloud index method and device, computer readable storage medium and terminal equipment - Google Patents

Point cloud index method and device, computer readable storage medium and terminal equipment Download PDF

Info

Publication number
CN114066994A
CN114066994A CN202111277617.4A CN202111277617A CN114066994A CN 114066994 A CN114066994 A CN 114066994A CN 202111277617 A CN202111277617 A CN 202111277617A CN 114066994 A CN114066994 A CN 114066994A
Authority
CN
China
Prior art keywords
point cloud
cloud data
ordered
index
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111277617.4A
Other languages
Chinese (zh)
Inventor
孙铁成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ubtech Robotics Corp
Original Assignee
Ubtech Robotics Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ubtech Robotics Corp filed Critical Ubtech Robotics Corp
Priority to CN202111277617.4A priority Critical patent/CN114066994A/en
Publication of CN114066994A publication Critical patent/CN114066994A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Abstract

The application belongs to the technical field of point cloud processing, and particularly relates to a point cloud indexing method, a point cloud indexing device, a computer-readable storage medium and terminal equipment. The method comprises the following steps: acquiring an RGBD image acquired by a preset depth camera; acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters; and establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data. According to the method and the device, a one-time index establishing process is utilized to replace a process of searching frame by frame in the traditional method, the time consumption of the searching process which is the most time-consuming in the point cloud deep learning is compressed to zero time cost, the training and reasoning efficiency of the point cloud deep learning is greatly improved, and the real-time requirements of various application scenes are met.

Description

Point cloud index method and device, computer readable storage medium and terminal equipment
Technical Field
The application belongs to the technical field of point cloud processing, and particularly relates to a point cloud indexing method, a point cloud indexing device, a computer-readable storage medium and terminal equipment.
Background
The fast real-time semantic perception of massive point clouds, such as semantic segmentation or classification, is a key technology for intelligent perception of application scenes of robot navigation positioning, automatic driving and the like. The current semantic understanding of massive point clouds based on a deep learning method faces a great challenge in real-time. For example, point cloud feature extraction based on a deep learning method generally aggregates neighborhood point sets through a k-nearest neighbor search algorithm and performs local feature extraction, which is very important for improving cloud semantic understanding performance. Although many acceleration algorithms for k-nearest neighbor search exist, such as a kd-tree, etc., when a large amount of point-by-frame nearest neighbor search is faced with a large amount of time overhead, the time overhead can greatly reduce the efficiency of semantic segmentation, and the real-time requirement cannot be met.
Disclosure of Invention
In view of this, embodiments of the present application provide a point cloud indexing method, an apparatus, a computer-readable storage medium, and a terminal device, so as to solve the problems that in the prior art, the processing efficiency of point cloud data is low, and the real-time requirement cannot be met.
A first aspect of an embodiment of the present application provides a point cloud indexing method, which may include:
acquiring an RGBD image acquired by a preset depth camera;
acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters;
and establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data.
In a specific implementation manner of the first aspect, each point cloud element in the ordered point cloud data corresponds to each pixel in the RGBD image one to one, and an arrangement manner of each point cloud element in the ordered point cloud data is consistent with an arrangement manner of each pixel in the RGBD image.
In a specific implementation manner of the first aspect, the indexing of the ordered point cloud data in the deep learning process includes downsampling indexing, and the establishing of the index of the ordered point cloud data in the deep learning process according to the arrangement manner of each point cloud element in the ordered point cloud data includes:
performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data;
and taking the index of each point cloud element in the down-sampling point cloud data corresponding to each point cloud element in the ordered point cloud data as the down-sampling index of the ordered point cloud data.
In a specific implementation manner of the first aspect, the index of the ordered point cloud data in the deep learning process includes a neighbor index, and the establishing the index of the ordered point cloud data in the deep learning process according to an arrangement manner of each point cloud element in the ordered point cloud data includes:
determining a sliding window area taking any point cloud element in the ordered point cloud data as a center;
taking the index of the point cloud element corresponding to the point cloud element in the sliding window area as the neighbor index of the point cloud element;
and traversing each point cloud element in the ordered point cloud data to obtain a neighbor index of the ordered point cloud data.
In a specific implementation manner of the first aspect, the indexing of the ordered point cloud data in the deep learning process includes an upsampling index, and the establishing of the index of the ordered point cloud data in the deep learning process according to an arrangement manner of each point cloud element in the ordered point cloud data includes:
performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data;
and taking the index of each point cloud element in the ordered point cloud data corresponding to each point cloud element in the down-sampling point cloud data as the up-sampling index of the ordered point cloud data.
In a specific implementation manner of the first aspect, the camera internal reference includes: the converting the RGBD image into ordered point cloud data according to the camera intrinsic parameters comprises:
for any pixel in the RGBD image, calculating a point cloud element corresponding to the pixel according to the following formula:
Figure BDA0003330030000000031
Figure BDA0003330030000000032
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) The position of the optical center of the depth camera is (u, v), the position of the pixel is (u, v), the distance corresponding to the pixel is (d), and the point cloud element corresponding to the pixel is (x, y, z);
and traversing each pixel in the RGBD image to obtain the ordered point cloud data.
A second aspect of an embodiment of the present application provides a point cloud indexing apparatus, which may include:
the image acquisition module is used for acquiring an RGBD image acquired by a preset depth camera;
the point cloud data conversion module is used for acquiring camera internal parameters of the depth camera and converting the RGBD image into ordered point cloud data according to the camera internal parameters;
and the index establishing module is used for establishing the index of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data.
In a specific implementation manner of the second aspect, each point cloud element in the ordered point cloud data corresponds to each pixel in the RGBD image one to one, and an arrangement manner of each point cloud element in the ordered point cloud data is consistent with an arrangement manner of each pixel in the RGBD image.
In a specific implementation manner of the second aspect, the indexing of the ordered point cloud data in the deep learning process may include downsampling, and the index establishing module may include:
the down-sampling index establishing unit is used for performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data; and taking the index of each point cloud element in the down-sampling point cloud data corresponding to each point cloud element in the ordered point cloud data as the down-sampling index of the ordered point cloud data.
In a specific implementation manner of the second aspect, the index of the ordered point cloud data in the deep learning process may include a neighbor index, and the index establishing module may include:
the neighbor index establishing unit is used for determining a sliding window area taking any point cloud element in the ordered point cloud data as a center; taking the index of the point cloud element corresponding to the point cloud element in the sliding window area as the neighbor index of the point cloud element; and traversing each point cloud element in the ordered point cloud data to obtain a neighbor index of the ordered point cloud data.
In a specific implementation manner of the second aspect, the index of the ordered point cloud data in the deep learning process may include an upsampling index, and the index establishing module may include:
the up-sampling index establishing unit is used for performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data; and taking the index of each point cloud element in the ordered point cloud data corresponding to each point cloud element in the down-sampling point cloud data as the up-sampling index of the ordered point cloud data.
In a specific implementation manner of the second aspect, the camera internal parameter may include: the point cloud data conversion module is specifically used for calculating a point cloud element corresponding to any pixel in the RGBD image according to the following formula:
Figure BDA0003330030000000041
Figure BDA0003330030000000042
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) The position of the optical center of the depth camera is (u, v), the position of the pixel is (u, v), the distance corresponding to the pixel is (d), and the point cloud element corresponding to the pixel is (x, y, z); and traversing each pixel in the RGBD image to obtain the ordered point cloud data.
A third aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of any one of the point cloud indexing methods described above.
A fourth aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of any one of the point cloud indexing methods when executing the computer program.
A fifth aspect of the embodiments of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to execute the steps of any one of the point cloud indexing methods described above.
Compared with the prior art, the embodiment of the application has the advantages that: the method includes the steps that RGBD images collected by a preset depth camera are obtained; acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters; and establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data. According to the method and the device, a one-time index establishing process is utilized to replace a process of searching frame by frame in the traditional method, the time consumption of the searching process which is the most time-consuming in the point cloud deep learning is compressed to be zero time cost, the training and reasoning efficiency of the point cloud deep learning is greatly improved, and the real-time requirements of various application scenes are met.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flowchart illustrating an embodiment of a point cloud indexing method according to an embodiment of the present disclosure;
FIG. 2 is a schematic illustration of a downsampled index;
FIG. 3 is a schematic diagram of a neighbor index;
FIG. 4 is a schematic illustration of an upsampling index;
FIG. 5 is a block diagram of an embodiment of a point cloud indexing device according to an embodiment of the present disclosure;
fig. 6 is a schematic block diagram of a terminal device in an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
Referring to fig. 1, an embodiment of a point cloud indexing method in an embodiment of the present application may include:
and S101, acquiring an RGBD image acquired by a preset depth camera.
In an embodiment of the present application, the RGBD image may include two images: one is an R (Red ) G (Green, Green) B (Blue ) three-channel color image and the other is a depth image. The depth image is similar to a grayscale image except that each pixel value thereof is the distance of the sensor from the object. Usually, the RGB image and the depth image are registered, so there is a one-to-one correspondence between the pixels.
Optionally, after the RGBD image is acquired, a series of pre-processing may be performed, which may include, but is not limited to, image scaling, depth value scaling, data type conversion, etc.
In the embodiment of the present application, the size of the resulting RGBD image is denoted as W × H, where W and H are generally set to an integer power of 2 or an integer multiple of a higher power of 2, such as an integer multiple of 64.
And S102, acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters.
And each point cloud element in the ordered point cloud data corresponds to each pixel in the RGBD image one to one, and the arrangement mode of each point cloud element in the ordered point cloud data is consistent with the arrangement mode of each pixel in the RGBD image.
The camera internal reference may include: taking any one pixel in the RGBD image as an example, the horizontal focal length, the vertical focal length, and the optical center position, the point cloud element corresponding to the pixel can be calculated according to the following formula:
Figure BDA0003330030000000071
Figure BDA0003330030000000072
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) The position of the optical center of the depth camera is (u, v) the position of the pixel, d the distance corresponding to the pixel, and (x, y, z) the point cloud element corresponding to the pixel.
And traversing each pixel (total W x H pixels) in the RGBD image according to the mode, so as to calculate point cloud elements corresponding to the pixels respectively, wherein the point cloud elements (total W x H point cloud elements) form the ordered point cloud data.
It should be noted that, when depth information cannot be obtained due to the limitation of a depth sensor or a special material object, a pixel value corresponding to a depth image is an invalid value, and the position of the invalid value changes randomly, so that if a point cloud element corresponding to the random invalid value is removed, the ordering of the point cloud is destroyed. Therefore, in the process of generating the point cloud, the last point cloud element is used for replacing the point cloud element corresponding to the invalid value, so that the ordering of the point cloud is guaranteed, and the validity and the authenticity of the point cloud element are also guaranteed. Specifically, if the first pixel value of the depth image is an invalid value, the corresponding point cloud element is set to (0,0, 0).
Step S103, establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data.
The index of the ordered point cloud data in the deep learning process may include, but is not limited to, a downsampled index, a neighbor index, and an upsampled index.
The point cloud semantic understanding deep learning network usually uses a coding and decoding structure to extract global features. The encoding process is generally a resolution reduction process, and aims to extract global semantic features, and at this time, downsampling of point cloud data is required. In the embodiment of the application, a down-sampling index of the ordered point cloud data can be established in a way of uniformly sampling the point cloud data.
Fig. 2 is a schematic diagram of the down-sampling index, and when the down-sampling index is established, the ordered point cloud data may be first down-sampled according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data.
The downsampling parameter may be set according to an actual situation, which is not specifically limited in this embodiment of the present application. For example, if the down-sampling parameter is 2, the down-sampling point cloud data has H/2 × W/2 point cloud elements in common.
Then, an index corresponding to each point cloud element in the ordered point cloud data to each point cloud element in the down-sampled point cloud data may be used as the down-sampled index of the ordered point cloud data.
Because the neighbor relation of the point cloud elements can mostly represent the neighbor relation between the three-dimensional points, the constraint approximation between the point cloud elements can be utilized to quickly extract the neighbor points of the point cloud. Namely, a sliding window with a fixed size is preset, and when the sliding window slides to a position, the contained point cloud elements are regarded as the neighboring points of the point cloud at the position.
Fig. 3 is a schematic diagram of the neighbor index, and when the neighbor index is established, for any point cloud element in the ordered point cloud data, a sliding window area with the point cloud element as a center may be determined first, and an index of the point cloud element corresponding to the sliding window area may be used as the neighbor index of the point cloud element.
The size of the sliding window area may be set according to actual conditions, which is not specifically limited in the embodiment of the present application. For example, the size of the sliding window region may be set to 5 × 5, and then 25 neighboring points may be obtained by the sliding window method, where the middle point is the point cloud element itself.
It should be noted that the sliding window area at the edge of the ordered point cloud data needs to be guaranteed not to slide out of the range of the ordered point cloud data, so as to guarantee that all indexes are valid indexes.
And traversing each point cloud element in the ordered point cloud data according to the mode to obtain the neighbor indexes corresponding to the point cloud elements respectively, thereby forming the neighbor indexes of the ordered point cloud data.
Decoders are commonly used in two-dimensional image depth learning networks, and the structure can effectively map global information to local detail regions. However, in the deep learning network of the point cloud, the structure is difficult to realize, which is mainly because of the disorder of the point cloud, and it is difficult to apply the nearest neighbor or interpolation method for up-sampling. To this end, the embodiment of the present application utilizes the ordering of the point clouds to establish an upsampling index of the ordered point cloud data.
Fig. 4 is a schematic diagram of the up-sampling index, and when the up-sampling index is established, the ordered point cloud data may be first down-sampled according to preset down-sampling parameters to obtain down-sampled point cloud data corresponding to the ordered point cloud data, and then an index corresponding to each point cloud element in the down-sampled point cloud data by each point cloud element in the ordered point cloud data is used as the up-sampling index of the ordered point cloud data.
In the same task, assuming that the resolution of the input RGBD image is kept unchanged, the index used in the point cloud deep learning network is kept unchanged, so a downsampling index, a neighbor index and an upsampling index can be extracted in advance according to the resolution of the input image. Only one-time index generation is needed, and the index can be repeatedly used in the subsequent deep learning process.
In summary, the RGBD image acquired by the preset depth camera is acquired in the embodiment of the present application; acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters; and establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data. According to the method and the device, a one-time index establishing process is utilized to replace a process of searching frame by frame in the traditional method, the time consumption of the searching process which is the most time-consuming in the point cloud deep learning is compressed to be zero time cost, the training and reasoning efficiency of the point cloud deep learning is greatly improved, and the real-time requirements of various application scenes are met.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Corresponding to the point cloud indexing method described in the foregoing embodiments, fig. 5 shows a structural diagram of an embodiment of a point cloud indexing device provided in the present application.
In this embodiment, a point cloud indexing device may include:
the image acquisition module 501 is configured to acquire an RGBD image acquired by a preset depth camera;
a point cloud data conversion module 502, configured to obtain camera parameters of the depth camera, and convert the RGBD image into ordered point cloud data according to the camera parameters;
the index establishing module 503 is configured to establish an index of the ordered point cloud data in the deep learning process according to the arrangement manner of each point cloud element in the ordered point cloud data.
In a specific implementation manner of the embodiment of the application, each point cloud element in the ordered point cloud data corresponds to each pixel in the RGBD image one to one, and an arrangement manner of each point cloud element in the ordered point cloud data is consistent with an arrangement manner of each pixel in the RGBD image.
In a specific implementation manner of the embodiment of the present application, the index of the ordered point cloud data in the deep learning process may include a down-sampling index, and the index establishing module may include:
the down-sampling index establishing unit is used for performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data; and taking the index of each point cloud element in the down-sampling point cloud data corresponding to each point cloud element in the ordered point cloud data as the down-sampling index of the ordered point cloud data.
In a specific implementation manner of the embodiment of the present application, the index of the ordered point cloud data in the deep learning process may include a neighbor index, and the index creating module may include:
the neighbor index establishing unit is used for determining a sliding window area taking any point cloud element in the ordered point cloud data as a center; taking the index of the point cloud element corresponding to the point cloud element in the sliding window area as the neighbor index of the point cloud element; and traversing each point cloud element in the ordered point cloud data to obtain a neighbor index of the ordered point cloud data.
In a specific implementation manner of the embodiment of the present application, the index of the ordered point cloud data in the deep learning process may include an upsampling index, and the index establishing module may include:
the up-sampling index establishing unit is used for performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data; and taking the index of each point cloud element in the ordered point cloud data corresponding to each point cloud element in the down-sampling point cloud data as the up-sampling index of the ordered point cloud data.
In a specific implementation manner of the embodiment of the present application, the camera internal reference may include: the point cloud data conversion module is specifically used for calculating a point cloud element corresponding to any pixel in the RGBD image according to the following formula:
Figure BDA0003330030000000111
Figure BDA0003330030000000112
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) The position of the optical center of the depth camera is (u, v), the position of the pixel is (u, v), the distance corresponding to the pixel is (d), and the point cloud element corresponding to the pixel is (x, y, z); and traversing each pixel in the RGBD image to obtain the ordered point cloud data.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Fig. 6 shows a schematic block diagram of a terminal device provided in an embodiment of the present application, and only shows a part related to the embodiment of the present application for convenience of description.
As shown in fig. 6, the terminal device 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62 stored in said memory 61 and executable on said processor 60. The processor 60 executes the computer program 62 to implement the steps in the above-mentioned various embodiments of the point cloud indexing method, such as the steps S101 to S103 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of each module/unit in the above-mentioned device embodiments, for example, the functions of the modules 501 to 503 shown in fig. 5.
Illustratively, the computer program 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 62 in the terminal device 6.
The terminal device 6 may be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a palm computer, a robot, or other computing devices. It will be understood by those skilled in the art that fig. 6 is only an example of the terminal device 6, and does not constitute a limitation to the terminal device 6, and may include more or less components than those shown, or combine some components, or different components, for example, the terminal device 6 may further include an input-output device, a network access device, a bus, etc.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or a memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used for storing the computer programs and other programs and data required by the terminal device 6. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (10)

1. A point cloud indexing method, comprising:
acquiring an RGBD image acquired by a preset depth camera;
acquiring camera internal parameters of the depth camera, and converting the RGBD image into ordered point cloud data according to the camera internal parameters;
and establishing indexes of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data.
2. The method of claim 1, wherein each point cloud element in the ordered point cloud data corresponds to each pixel in the RGBD image one-to-one, and the arrangement of each point cloud element in the ordered point cloud data is consistent with the arrangement of each pixel in the RGBD image.
3. The point cloud indexing method of claim 1, wherein the index of the ordered point cloud data in the deep learning process comprises a down-sampling index, and the establishing of the index of the ordered point cloud data in the deep learning process according to the arrangement of each point cloud element in the ordered point cloud data comprises:
performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data;
and taking the index of each point cloud element in the down-sampling point cloud data corresponding to each point cloud element in the ordered point cloud data as the down-sampling index of the ordered point cloud data.
4. The point cloud indexing method according to claim 1, wherein the index of the ordered point cloud data in the deep learning process comprises a neighbor index, and the establishing of the index of the ordered point cloud data in the deep learning process according to the arrangement of each point cloud element in the ordered point cloud data comprises:
determining a sliding window area taking any point cloud element in the ordered point cloud data as a center;
taking the index of the point cloud element corresponding to the point cloud element in the sliding window area as the neighbor index of the point cloud element;
and traversing each point cloud element in the ordered point cloud data to obtain a neighbor index of the ordered point cloud data.
5. The point cloud indexing method of claim 1, wherein the index of the ordered point cloud data in the deep learning process comprises an upsampling index, and the establishing of the index of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data comprises:
performing down-sampling operation on the ordered point cloud data according to preset down-sampling parameters to obtain down-sampling point cloud data corresponding to the ordered point cloud data;
and taking the index of each point cloud element in the ordered point cloud data corresponding to each point cloud element in the down-sampling point cloud data as the up-sampling index of the ordered point cloud data.
6. The point cloud indexing method of any one of claims 1 to 5, wherein the camera parameters include: the converting the RGBD image into ordered point cloud data according to the camera intrinsic parameters comprises:
for any pixel in the RGBD image, calculating a point cloud element corresponding to the pixel according to the following formula:
Figure FDA0003330029990000021
Figure FDA0003330029990000022
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) Is the optical center position of the depth camera, (u, v) is the position of the pixel, and d is the pixel pairThe corresponding distance (x, y, z) is the point cloud element corresponding to the pixel;
and traversing each pixel in the RGBD image to obtain the ordered point cloud data.
7. A point cloud indexing device, comprising:
the image acquisition module is used for acquiring an RGBD image acquired by a preset depth camera;
the point cloud data conversion module is used for acquiring camera internal parameters of the depth camera and converting the RGBD image into ordered point cloud data according to the camera internal parameters;
and the index establishing module is used for establishing the index of the ordered point cloud data in the deep learning process according to the arrangement mode of each point cloud element in the ordered point cloud data.
8. The point cloud indexing device of claim 7, wherein the camera reference comprises: the point cloud data conversion module is specifically used for calculating a point cloud element corresponding to any pixel in the RGBD image according to the following formula:
Figure FDA0003330029990000031
Figure FDA0003330029990000032
z＝d
wherein f isxIs the horizontal focal length of the depth camera, fyIs the vertical focal length of the depth camera, (c)x,cy) The position of the optical center of the depth camera is (u, v), the position of the pixel is (u, v), the distance corresponding to the pixel is (d), and the point cloud element corresponding to the pixel is (x, y, z); and traversing each pixel in the RGBD image to obtain the ordered point cloud data.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the point cloud indexing method of any one of claims 1 to 6.
10. Terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor realizes the steps of the point cloud indexing method according to any of claims 1 to 6 when executing the computer program.
CN202111277617.4A 2021-10-29 2021-10-29 Point cloud index method and device, computer readable storage medium and terminal equipment Pending CN114066994A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111277617.4A CN114066994A (en) 2021-10-29 2021-10-29 Point cloud index method and device, computer readable storage medium and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111277617.4A CN114066994A (en) 2021-10-29 2021-10-29 Point cloud index method and device, computer readable storage medium and terminal equipment

Publications (1)

Publication Number Publication Date
CN114066994A true CN114066994A (en) 2022-02-18

Family

ID=80236136

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111277617.4A Pending CN114066994A (en) 2021-10-29 2021-10-29 Point cloud index method and device, computer readable storage medium and terminal equipment

Country Status (1)

Country Link
CN (1) CN114066994A (en)

Similar Documents

Publication Publication Date Title
CN110717851A (en) Image processing method and device, neural network training method and storage medium
CN111079764B (en) Low-illumination license plate image recognition method and device based on deep learning
CN105069042A (en) Content-based data retrieval methods for unmanned aerial vehicle spying images
CN110991310B (en) Portrait detection method, device, electronic equipment and computer readable medium
CN111402170A (en) Image enhancement method, device, terminal and computer readable storage medium
CN112200115B (en) Face recognition training method, recognition method, device, equipment and storage medium
Damavandi et al. Speed limit traffic sign detection and recognition
CN102663399A (en) Image local feature extracting method on basis of Hilbert curve and LBP (length between perpendiculars)
CN112926548A (en) Lane line detection method and device, electronic equipment and storage medium
CN111191582A (en) Three-dimensional target detection method, detection device, terminal device and computer-readable storage medium
CN112651953A (en) Image similarity calculation method and device, computer equipment and storage medium
CN111563517A (en) Image processing method, image processing device, electronic equipment and storage medium
CN108877030B (en) Image processing method, device, terminal and computer readable storage medium
CN108805838B (en) Image processing method, mobile terminal and computer readable storage medium
CN114359048A (en) Image data enhancement method and device, terminal equipment and storage medium
CN114444565A (en) Image tampering detection method, terminal device and storage medium
CN113570725A (en) Three-dimensional surface reconstruction method and device based on clustering, server and storage medium
CN113506305A (en) Image enhancement method, semantic segmentation method and device for three-dimensional point cloud data
CN112380978A (en) Multi-face detection method, system and storage medium based on key point positioning
CN114519788A (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
CN114066994A (en) Point cloud index method and device, computer readable storage medium and terminal equipment
CN110555406A (en) Video moving target identification method based on Haar-like characteristics and CNN matching
CN111126248A (en) Method and device for identifying shielded vehicle
CN114511862B (en) Form identification method and device and electronic equipment
CN113947529B (en) Image enhancement method, model training method, component identification method and related equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination