EP4244815A2 - Aufgabenbewusste punktwolken-downsampling - Google Patents
Aufgabenbewusste punktwolken-downsamplingInfo
- Publication number
- EP4244815A2 EP4244815A2 EP21835452.0A EP21835452A EP4244815A2 EP 4244815 A2 EP4244815 A2 EP 4244815A2 EP 21835452 A EP21835452 A EP 21835452A EP 4244815 A2 EP4244815 A2 EP 4244815A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- point cloud
- point
- level feature
- feature vector
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/004—Predictors, e.g. intraframe, interframe coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/56—Particle system, point based geometry or rendering
Definitions
- the present principles generally relate to the domain of point cloud processing.
- the present document is also understood in the context of the analysis, the interpolation, the representation and the understanding of point cloud signals.
- Point cloud is a data format used across several business domains including autonomous driving, robotics, AR/VR, civil engineering, computer graphics, and the animation/movie industry.
- 3D LIDAR sensors have been deployed in self-driving cars, and affordable LIDAR sensors are included with, for example, Apple iPad Pro 2020 and Intel Real Sense LIDAR camera L515.
- 3D point cloud data has become more practical and is expected to be a valuable enabler in the applications mentioned.
- point cloud data may consume a large portion of network traffic, e.g., among connected cars over a 5G network, and immersive communications (virtual or augmented reality (VR/AR)).
- VR/AR virtual or augmented reality
- Point cloud understanding and communication would essentially lead to efficient representation formats.
- raw point cloud data need to be properly organized and processed for the purposes of world modeling and sensing.
- point clouds may represent a sequential scan of the same scene, which contains multiple moving objects. These are called dynamic point clouds as compared to static point clouds captured from a static scene or static objects. Dynamic point clouds are typically organized into frames, with different frames being captured at different time.
- 3D point cloud data are essentially discrete samples of the surfaces of objects or scenes.
- a large number of points is required.
- a typical VR immersive scene contains millions of points
- point cloud maps typically contain hundreds of millions of points. Therefore, the processing of such large- scale point clouds is computationally expensive, especially for consumer devices that have limited computational power, e.g., smartphones, tablets, and automotive navigation systems.
- Point cloud data are key for various applications, such as autonomous driving, VR/AR, topography and cartography, etc.
- consuming a large point cloud directly incurs significant computational costs. Consequently, it is important to adaptively down-sample the input point cloud to facilitate subsequent tasks.
- Such a down-sampling process is useful for scene-flow estimation, point cloud compression, and other general computer vision tasks.
- the present principles relate to a method that generates, using a neural network, a pointlevel feature vector for each point of a point cloud and a set-level feature vector for the point cloud.
- a representative position based on the point-level feature vectors and on the set-level feature vector is generated.
- the representative position and the set-level feature vector are output as a set descriptor.
- a method for retrieving a point cloud from a data stream obtains, from the data stream, a down-sampled point cloud and a residual point cloud.
- the down-sampled point cloud is fed to a predictor construction module to obtain a predicted point cloud.
- the point cloud is retrieved by adding the predicted point cloud to the residual point cloud.
- the present principles also relate to device comprising at least one processor associated with at least one memory configured to implement embodiments corresponding to the methods above.
- Fig- 1 illustrates a method 10 of down-sampling an input point cloud X with n points for subsequent machine tasks, according to a non-limiting embodiment of the present principles
- Fig. 2 diagrammatically illustrates the SD function, according to a non-limiting embodiment of the present principles
- Fig- 3 illustrates an example, where a point A is chosen as the representative point because it has the largest weight
- Fig- 4 illustrates a fifth embodiment of down-sampling an input point cloud according to the present principles
- Fig. 5 diagrammatically illustrates how to integrate the task-aware point cloud downsampling method of the present principles with a sub-sequent machine task
- Fig. 6 illustrates a seventh embodiment of an integrated task-aware point cloud downsampling method
- Fig. 7 illustrates a method of point cloud compression using an embodiment of a task- aware point cloud down-sampling method according to the present principles
- Fig. 8 illustrates a decoder embodiment of the present principles
- Fig. 9 shows an example architecture of a device 30 which may be configured to implement a method described in relation to Fig. 1. 5. Detailed description of embodiments
- each block represents a circuit element, module, or portion of code which comprises one or more executable instructions for implementing the specified logical function(s).
- the function(s) noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending on the functionality involved.
- LIDARs produce (dynamic) point clouds that are used by a decision engine. These point clouds are not intended to be viewed by human eyes and they are typically sparse, not necessarily colored, and dynamic with a high frequency of capture. They may have other attributes like the reflectance ratio provided by the LIDAR as this attribute is indicative of the material of the sensed object and may help in making a decision.
- VR Virtual Reality
- immersive worlds have become widely discussed, foreseen by many as the future of 2D flat video.
- the basic idea is to immerse the viewer in an environment all around him as opposed to standard TV where he only views the virtual world in front of him.
- Point cloud is a good format candidate to distribute VR worlds. They may be static or dynamic and are typically of average size, for example, no more than millions of points at a time.
- Point clouds may be also used for various purposes such as cultural heritage/buildings in which objects like statues or buildings are scanned in 3D in order to share the spatial configuration of the object without sending or physically visiting it. This also provides a way to preserve the information and data about the object in case it may be destroyed; for instance, a temple by an earthquake. Such point clouds are typically static, colored, and relatively large.
- 3D representations and maps are not limited to a plane and may include relief features.
- Google Maps is one example of 3D maps that uses meshes instead of point clouds. Nevertheless, point clouds may be a suitable data format for 3D maps and such point clouds are typically static, colored, and relatively large.
- World modeling & sensing via point clouds could be a technology to allow machines to gain knowledge about the 3D world around them, helpful for the applications discussed above.
- 3D point cloud data are essentially discrete samples of the surfaces of objects or scenes. To fully represent the real world with point samples, in practice, a large number of points is required. Therefore, the processing of such large-scale point clouds is computationally expensive, especially for consumer devices that have limited computational power, e.g., smartphones, tablets, and automotive navigation systems.
- one solution is to down-sample it first, where the down-sampled point cloud summarizes the geometry of the input point cloud while having significantly fewer points.
- the down-sampled point cloud is then fed to the subsequent machine task for further consumption.
- point cloud data can be exploited for various tasks, such as scene flow estimation, classification, detection, segmentation, and compression, etc.
- Different tasks focus on different aspects of a point cloud. For instance, classification relies on the saliency points of the geometry, while object segmentation needs to distinguish the points on one object from the others, and scene flow estimation counts the dynamics of a point cloud.
- an adaptive point cloud down-sampling algorithm that is task-aware is helpful. Therefore, when faced with different tasks, the same point cloud can be down-sampled to different ones to facilitate the subsequent tasks.
- Fig- 1 illustrates a method 10 of down-sampling an input point cloud X with n points for subsequent machine tasks according to the present principles.
- an initial down-sampled point cloud with m points (m ⁇ ri) is selected.
- a set of m points, like point 110, of the input point cloud is selected using any applicable method.
- anchor point for a point 110 in the initial down- sampled point cloud (herein called the “anchor point”), its nearby points are aggregated from the point cloud X, leading to a local point set 120. In this way, each anchor point in the initial down- sampled point cloud is associated with a local point set from the point cloud X.
- each point set is fed to a module herein called the Set Distillation (SD) function, resulting in a representative point 130 and its corresponding set-level feature.
- SD Set Distillation
- the SD function first computes a point-level feature vector for each point in the point set, and a set-level feature vector describing the overall point set. This step is accomplished, for example, using a neural network module (herein called P-Net) structured according to the present principles.
- P-Net neural network module
- a representative position is computed. This step is achieved through either a deterministic approach or another neural network module.
- the SD function outputs the representative position, as well as the set-level feature to represent the geometry of the point set.
- the obtained representative position is not limited to the points within the point set.
- the m representative points are aggregated as the updated down-sampled point cloud, which is fed to the subsequent task for further processing.
- the m set-level features are also optionally output and fed to the subsequent task.
- Down-sampling method 10 is integrated with the subsequent task and trained in an end-to- end manner, allowing down-sampling method 10 be task-aware, i.e., adaptive to the machine task.
- the down-sampled point clouds obtained by method 10 are able to capture the underlying geometry for a particular machine task, regardless of the how original input point cloud is sampled from the scene.
- method 10 results in two down-sampled point clouds that are closely resemble each other.
- Fig. 2 diagrammatically illustrates an example of the SD function.
- the SD function feeds the point set to a PointNet architecture as described, for example, in “PointNet: Deep learning on point sets for 3D classification and segmentation,” in proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 652-660, 2017, by C.R. Qi, H. Su, K. Mo, L.J. Guibas.
- a module 21 of PointNet computes point-level feature vectors 22 for each point with shared multi-layer perception (MLP). These point-level feature vectors 22 are then aggregated with a max-pooling operation 23, resulting in a set-level feature vector 24 describing the whole point set.
- MLP multi-layer perception
- a set of weights 26 is computed for the points in the whole point set.
- an affinity value e.g., weight estimate
- This affinity value describes the of the degree to which its associated point is representative of the whole point set.
- a module 27 performs a weighted average 28 of the points with the computed weights 26, to generate a representative position for the point set.
- the affinity values are converted to a set of weights using the Softmax(-) function, so that all the weight values are greater than 0 and summed up to 1.
- a weighted averaging of the x coordinates of all points in the point set with the obtained weights is performed, leading to the x coordinate of the generated representative point.
- the y and z coordinates of the representative point are computed with the weights.
- the generated x, y and z coordinates form the position of the representative point.
- the SD function outputs the representative point, as well as the set-level feature generated by PointNet.
- the down-sampling of a given point cloud X containing n points is performed using the presented SD function.
- an initial down- sampled point cloud with m points is generated using the Farthest point sampling (FPS) method, where the obtained points are called the “anchor points”.
- FPS Farthest point sampling
- Anchor points is a known point cloud down-sampling approach and is described for instance in “The Farthest point strategy for progressive image sampling,” IEEE Trans, on Image Processing, vol. 6, no. 9, pp. 1306-1315, 1997.
- FPS is based on repeatedly choosing the next sample point in the least-explored area.
- the FPS algorithm chooses the farthest point to the subset from the rest of the points in X with some distance measure. This farthest point is then added to the subset.
- the subset is initialized by randomly picking a point from X.
- the FPS algorithm repeats this point selection process until a certain condition is met, e.g., the number of points in the subset reaches a predefined threshold. This classic sampling approach is deterministic and does not consider the downstream task.
- each anchor point its nearby points are collected through a ball query procedure, i.e., all points in X lying within a predefined distance r to the anchor point are identified and collected, forming a local point set for that anchor point.
- every local point set (m in total) is fed to the SD function individually, leading to the updated down-sampled point cloud (with m points), accompanied with m set-level features.
- the m down-sampled points (and optionally, the set level features) are fed to the subsequent task.
- This down-sampling method is trained end-to-end with the subsequent machine task, to make the neural network layers in the SD function task-aware, i.e., be adaptive to the subsequent task.
- the computation of the point-wise weights of the SD function differs.
- a distance is computed for each point in the point set, which is the Euclidean distance between its point-level feature vector, and the setlevel feature vector.
- the weight values of the point set are further normalized so that they summed up to one.
- a weighted averaging of the points in the point set is performed using the obtained weights, as presented in relation to the first embodiment, leading to the representative point position.
- the SD function returns the representative point as well as the set-level feature obtained by the PointNet.
- each down-sampled point is obtained by selecting a critical point in a local point set.
- a difference between this embodiment and the first embodiment lies in the SD function, where the SD function here chooses a representative point from the input point set. Similar to the first embodiment, given a point set, the SD function computes a set of weights for each point in the whole set. Then the SD function directly returns the point with the maximum weight as the representative point, as well as returns the set-level feature generated by PointNet.
- Fig- 3 illustrates an example, where a point A is chosen as the representative point because it has the largest weight.
- the method in the second embodiment to compute the weights for the points in the point set, where the weights are obtained through a Gaussian kernel may also be used.
- the SD function takes as inputs not only a local point set from the point cloud X but also a one-hot vector indicating which point is the anchor point of the point set. Consequently, the SD function in this embodiment can utilize the knowledge of the anchor point position to generate the representative point.
- the position vector of each point in the point set is augmented, by appending the position vector of the anchor point (and the feature vector of the anchor, which is another input to the SD function, if available). With the information of the anchor position appended, the augmented point set is then processed by the PointNet, leading to the point-level feature vectors of each point and the set-level feature vector.
- Fig- 4 illustrates a fifth embodiment of down-sampling an input point cloud according to the present principles.
- this embodiment directly modifies the position of the anchor point 41, then returns the modified position 42 as the representative point.
- the SD function in this embodiment also takes as inputs a local point set 20 as well as a one-hot vector indicating which point is the anchor point 41 of the point set.
- the point-level feature vectors 22 and the set-level feature 24 are obtained, they are fed to another neural network 43, herein called the “M-Net.”
- the M-Net specifically outputs a modification vector 44 relative to the anchor point position; it may be implemented with a PointNet architecture.
- the representative point position 42 is obtained by adding the modification vector 44 and the anchor position 41.
- the SD function still returns the representative point and the set-level feature vector.
- the fifth embodiment can be combined with the fourth embodiment where the points fed to the SD function are first augmented by the information of the anchor position.
- Fig. 5 diagrammatically illustrates how to integrate the task-aware point cloud downsampling method of the present principles with a sub-sequent machine task. As an example, the task of scene flow estimation for the 3D point cloud is considered, without loss of generality, for illustrating this sixth embodiment.
- output scene flow 53 includes a set of 3D vectors, where each 3D vector is associated with a point of the first point cloud frame.
- the 3D vectors describe how the points from the first point cloud frame physically move to the surface of the second point cloud frame.
- the scene flow between two-point cloud frames describes the dynamics of the point clouds, which is essential for many practical applications, e.g., autonomous driving, AR/VR, and robotics.
- the down-sampling methods presented in the previous embodiments are applied multiple times.
- the overall neural network architecture of this embodiment takes an hour-glass structure with skip connections.
- the method of the present sixth embodiment comprises a first stage generating a first and a second down-sampled point clouds, from the first and the second point cloud frames, respectively. This is achieved using two task- aware down-sampling modules 54a and 54b (based on any one of the previous embodiments) for both inputs.
- Two consecutive point clouds of a point cloud sequence may be considered as one point cloud in which points carry a temporal information indicating whether they belong to the first or to the second point cloud. Indeed, two point clouds of a sequence of point clouds share the same frame of reference and their points can be merged in one point cloud.
- a point set for each point in the first down-sampled point cloud is aggregated by searching for its nearest-neighboring points from the second down-sampled point cloud.
- the method computes a first inter-frame feature fusing the information from both point cloud frames, for each point in the first down-sampled point cloud, using the information of the point (its position and the point-level feature) as well as its associated nearest-neighboring point set.
- This second stage is accomplished using a neural network module 55, herein called “Fl-Net”.
- the first down-sampled point cloud is further down-sampled with a task-aware down-sampling module 54c according to the present principles, taking the points and the associated inter-frame features as inputs.
- a second inter-frame feature is computed for each point in the first point cloud frame, using an up-sampled neural network module 56, herein called “F2-Net”.
- This F2-Net module corresponds to stacks of Set Up-Conv layers, for example as presented in “FlowNet3D: Learning scene flow in 3D point clouds,” in proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 529-537, 2020.
- Such a neural network module interpolates the point-wise features hierarchically.
- a scene flow vector is computed for each point in the first point cloud frame, using a feature-to-flow transformation neural network module 57, herein called “F3- Ne ’.
- F3-Net is implemented with pointwise MLP layers. According to the present principles, skipconnections between the task-aware down-sampling module and the F2-Net are added to merge information from the early layers.
- EPE end-point-error
- Fig- 6 illustrates a seventh embodiment of an integrated task-aware point cloud downsampling method.
- This seventh embodiment estimates a scene flow 53 based on two input point cloud frames 51 and 52.
- the method iteratively updates the estimated scene flow to refine its accuracy through a flow interpolation module 61.
- An initial point- wise scene flow for the first point cloud frame 51 is estimated using the method described above with respect to Fig. 5.
- a point-wise scene flow is generated for the first down-sampled point cloud. This is to achieve based on a scene flow interpolation neural network module 61, herein called “I-Net”.
- I-Net is implemented in the same manner as a Set Up-Conv layer as described in “FlowNet3D”.
- a shifted down-sampled point cloud is generated, via shifting each point in the first down-sampled point cloud by its associated scene flow vector.
- a point set is then aggregated for each point in the shifted down-sampled point cloud, by searching for its nearest-neighboring points from the second down-sampled point cloud.
- each point in the first down-sampled point cloud is associated with its shifted version, as well as an updated nearest-neighboring point set using the shifted version as the query point.
- a second point-wise scene flow for the first point cloud frame by executing Fl-Net 55, the second task-aware down-sampling module 54c, F2-Net 56, and F3-Net 57 again.
- An alternative of this stage is that, based on the shifted down-sampled point cloud, and the updated nearest-neighboring point sets, executing Fl -Net, the second task-aware down-sampling module, F2-Net, and F3-Net again, leading to a residual point-wise scene flow.
- a second point-wise scene flow for the first point cloud frame can be obtained.
- the second point-wise scene flow is output as the result.
- This recurrent scene flow estimation scheme can be executed iteratively for more than two iterations until a certain condition is satisfied, e.g., the number of iterations reaches a predefined threshold.
- Fig- 7 illustrates a method of point cloud compression using an embodiment of a task- aware point cloud down-sampling method according to the present principles.
- the down-sampled point cloud is used to construct a predicted point cloud for a predictive coding task.
- a down-sampled point cloud is generated using a task-aware point cloud down-sampling method 71 as described in relation to one of the previous embodiments.
- the down-sampled point cloud, and optionally, the generated set-level feature vectors are on one hand, encoded by a first entropy encoder 72 leading to a first bit-stream BSr, while on the other hand, the down-sampled point cloud, and optionally, the generated set-level feature vectors are fed to a predictor construction module 73 which endeavors to generate a predicted point cloud X P that is close to X.
- the two bitstreams together are sent to the decoder.
- the entropy encoders 72 and 74 can either be lossless or lossy.
- Fig- 8 illustrates a decoder embodiment of the present principles.
- the down-sampled point cloud (and the feature vectors if available) is decoded from the first bit-stream BS1 by a decoder module 81 and fed to a predictor construction module 82 to obtain the predicted point cloud X P .
- the residual point cloud X R from the second bit-stream BS2 is decoded by a decoder module 83. By adding up X P and X R , the reconstructed point cloud X is obtained.
- This decoder embodiment can be used for either inter-frame predictive coding or intraframe predictive coding. It differs from conventional scalable coding in two aspects. On one hand, the decoder does not limit the down-sampled point cloud to be a subset of the input point cloud, On the other hand, aside from the down-sampled point cloud, the feature vectors produced the task-aware down-sampling module of the encoder in relation to Fig. 7 can also be employed to generate the predicted point cloud, which gives the present predictive coding scheme more flexibility.
- Fig- 9 shows an example architecture of a device 30 which may be configured to implement a method described in relation Figs. 1, 5, 6, 7, and 8.
- the different embodiments of encoders and decoders according to the present principles may implement this architecture.
- each module of encoders and/or decoders according to the present principles may be a device according to the architecture of Fig. 9, linked together, for instance, via their bus 31 and/or via I/O interface 36.
- Device 30 comprises the following elements that are linked together by a data and address bus 31 :
- a microprocessor 32 which is, for example, a DSP (or Digital Signal Processor);
- a power supply e.g., a battery (not shown).
- the power supply is external to the device.
- the word « register » used in the specification may correspond to an area of small capacity (some bits) or to very large area (e.g. a whole program or large amount of received or decoded data).
- the ROM 33 comprises at least a program and parameters. The ROM 33 may store algorithms and instructions to perform techniques in accordance with present principles. When switched on, the CPU 32 uploads the program in the RAM and executes the corresponding instructions.
- the RAM 34 comprises, in a register, the program executed by the CPU 32 and uploaded after switch-on of the device 30, input data in a register, intermediate data in different states of the method in a register, and other variables used for the execution of the method in a register.
- the implementations described herein may be implemented in, for example, a method or a process, an apparatus, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program).
- An apparatus may be implemented in, for example, appropriate hardware, software, and firmware.
- the methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs”), and other devices that facilitate communication of information between end-users.
- PDAs portable/personal digital assistants
- the device 30 belongs to a set comprising:
- a server e.g., a broadcast server, a video-on-demand server or a web server.
- Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and/or depth information.
- equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices.
- the equipment may be mobile and even installed in a mobile vehicle.
- the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”).
- the instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination.
- a processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.
- implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted.
- the information may include, for example, instructions for performing a method, or data produced by one of the described implementations.
- a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment.
- Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal.
- the formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream.
- the information that the signal carries may be, for example, analog or digital information.
- the signal may be transmitted over a variety of different wired or wireless links, as is known.
- the signal may be stored on a processor-readable medium.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Character Discrimination (AREA)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063113129P | 2020-11-12 | 2020-11-12 | |
| PCT/US2021/059074 WO2022104012A2 (en) | 2020-11-12 | 2021-11-12 | Task-aware point cloud down-sampling |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4244815A2 true EP4244815A2 (de) | 2023-09-20 |
Family
ID=79170934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP21835452.0A Pending EP4244815A2 (de) | 2020-11-12 | 2021-11-12 | Aufgabenbewusste punktwolken-downsampling |
Country Status (5)
| Country | Link |
|---|---|
| EP (1) | EP4244815A2 (de) |
| JP (1) | JP2023549320A (de) |
| CN (1) | CN116569219A (de) |
| MX (1) | MX2023005028A (de) |
| WO (1) | WO2022104012A2 (de) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188924B (zh) * | 2022-12-20 | 2026-02-06 | 辽宁工程技术大学 | 一种基于多尺度特征提取的点云补全方法 |
| WO2025112055A1 (zh) * | 2023-12-01 | 2025-06-05 | Oppo广东移动通信有限公司 | 点云数据处理方法、装置、设备以及存储介质 |
| WO2026016087A1 (zh) * | 2024-07-17 | 2026-01-22 | Oppo广东移动通信有限公司 | 点云属性解码、编码方法、编解码器及介质 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180053324A1 (en) * | 2016-08-19 | 2018-02-22 | Mitsubishi Electric Research Laboratories, Inc. | Method for Predictive Coding of Point Cloud Geometries |
| US10911787B2 (en) * | 2018-07-10 | 2021-02-02 | Apple Inc. | Hierarchical point cloud compression |
-
2021
- 2021-11-12 WO PCT/US2021/059074 patent/WO2022104012A2/en not_active Ceased
- 2021-11-12 MX MX2023005028A patent/MX2023005028A/es unknown
- 2021-11-12 CN CN202180076586.8A patent/CN116569219A/zh active Pending
- 2021-11-12 EP EP21835452.0A patent/EP4244815A2/de active Pending
- 2021-11-12 JP JP2023525557A patent/JP2023549320A/ja active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN116569219A (zh) | 2023-08-08 |
| WO2022104012A3 (en) | 2022-07-21 |
| JP2023549320A (ja) | 2023-11-24 |
| MX2023005028A (es) | 2023-07-05 |
| WO2022104012A2 (en) | 2022-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20250139834A1 (en) | Scalable framework for point cloud compression | |
| US20230410254A1 (en) | Task-aware point cloud down-sampling | |
| Xiao et al. | Knowledge-based coding of objects for multisource surveillance video data | |
| EP4244815A2 (de) | Aufgabenbewusste punktwolken-downsampling | |
| KR20230131878A (ko) | 포인트 클라우드 프로세싱을 위한 장치 및 방법 | |
| EP4606099A1 (de) | Baumbasiertes tiefes entropiemodell zur punktwolkenkompression | |
| US20230419609A1 (en) | Self-supervised 3d point cloud abstraction | |
| US20250200815A1 (en) | Unsupervised 3d point cloud distillation and segmentation | |
| US20250045971A1 (en) | Hybrid framework for point cloud compression | |
| US11915373B1 (en) | Attribute value compression for a three-dimensional mesh using geometry information to guide prediction | |
| Wang et al. | suLPCC: a novel LiDAR point cloud compression framework for scene understanding tasks | |
| KR20250137634A (ko) | 동적 포인트 클라우드들에 대한 학습 기반 예측 코딩 | |
| US12518433B2 (en) | Learning-based point cloud compression via tearing transform | |
| US20250316038A1 (en) | Point cloud decoder with 6d pose estimation | |
| US20240282013A1 (en) | Learning-based point cloud compression via unfolding of 3d point clouds | |
| US20240406427A1 (en) | Method and apparatus for point cloud compression using hybrid deep entropy coding | |
| Wang et al. | Towards Point Cloud Geometry Compression via Global-local and Multi-scale Feature Learning | |
| US20260044933A1 (en) | Method, apparatus, and medium for point cloud processing | |
| WO2025184001A1 (en) | Learning-based image coding for 3d point cloud data, with each color component of a 2d image corresponding to a dimension of 3d cartesian coordinates of a point position | |
| Arubai et al. | Building a Real‐Time 2D Lidar Using Deep Learning | |
| Gao | Fundamentals for Deep Learning-based Image and Video Coding | |
| KR20240115237A (ko) | 아웃라이어 그룹핑 기반 포인트 클라우드 압축 | |
| KR20260052126A (ko) | 베이스 그래프 기반 메시 압축 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20230503 |
|
| AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) |