Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/50—Depth or shape recovery
  • G06T7/529—Depth or shape recovery from texture
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
  • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/04—Texture mapping
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/10—Geometric effects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/10—Geometric effects
  • G06T15/20—Perspective computation
  • G06T15/205—Image-based rendering
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2210/00—Indexing scheme for image generation or computer graphics
  • G06T2210/56—Particle system, point based geometry or rendering

Definitions

• a volume unit being associated with a point of the 3D scene, the depth difference corresponding to a number of volume units, the number of generated points corresponding to the depth difference minus 1 .
• attributes to be associated with the at least one additional point are determined, the attributes being determined from attributes associated with the current point and with the adjacent pixel.
• the points of the 3D scene are part of a point cloud.
• FIG. 13 shows an example of a process for decoding a bitstream to obtain the decoded point cloud representing the 3D object of figure 1 object of figure 1 , in accordance with a non-limiting embodiment of the present principles.
• the part of each image that receives attributes from the point cloud is shown as a grey area while the part of the image that does not receive attributes from the point cloud is shown as a white area, said white area may be filled with default value, like the free space between images.
• the data associated with the pixels of the images 21 1 to 21 n may correspond to texture information and/or depth information.
• a first picture 21 is used to store the texture information (e.g. 3 components RGB or YUV) and a second picture 21 with the same arrangement of images 21 1 to 21 m is used to store the depth information, both pictures representing the point cloud at time 't ⁇
• LLE (Locally-Linear Embedding) that corresponds to a mathematical operation of dimension reduction, here applied to convert/transform from 3D to 2D, the parameters representative of the LLE comprising the transformation coefficients.
• Each image has advantageously a rectangular shape to ease the packing process on the picture 21 .
• volume elements different from the square may be associated with the points of the 3D scene, e.g. a sphere.
• the expression "volume unit” will be used to express the volume element associated with a point, a volume unit corresponding for example a voxel of size 1 by 1 by 1 , e.g. 1 mm by 1 mm by 1 mm (with a volume of 1 mm 3 ), or 1 cm by 1 cm by 1 cm (with a volume of 1 cm 3 ) or any other dimensions.
• the cubes/points 802 to 809 correspond to the neighborhood of the cube/point 601 as their corresponding pixels 612 to 619 of the associated depth image correspond to the adjacent pixels of the depth image surrounding the pixel 61 1 corresponding to the cube/point 801 .
• the depth image associated with the 3D scene may be used to determine where hoie(s) may be located in area(s) in the 3D scene.
• the part 8B of the depth image associated with (and obtained from) the part 6A of the points of the 3D object 10 is processed and analysed as explained hereinbelow to obtain the location of the hole(s).
• the depth information associated with the pixels 81 1 to 819 is used to obtain the location of the hole(s) 6001 , 6002.
• the block 6C of Figure 6B shows the depth information which is associated with the pixels 81 1 to 619.
• each point of the 3D scene may be processed as a current point and its depth compared with the depth of its neighborhood (i.e. in the space of the associated depth image).
• additional cubes/points may be generated between two cubes/points having a depth difference d (in the depth image) fulfilling the equation 1 .
• the additional cubes/points may be generating by computing their associated depth and texture from the depth and texture associated with the cubes used to determine the hole (e.g. by interpolation of the points/cubes used to determine the presence of a hole).
• the number of generated additional points may be a function of the depth difference, for example equals to d minus 1 (d - 1 ), when the depth difference is expressed with a number of volume units.
• the weight associated with a texture value when interpolating a texture value to be associated with a generated additional point may be inversely proportional to the distance (depth) separating the generating additional point from the point used to generate it.
• a weight equal to 2 may be associated with the texture of the point 601 and a weight equal to 1 may be associated with the texture of the point 604, the distance (depth difference) between the additional point 6001 and the point 601 being equal to 1 volume unit while the distance (depth difference) between the additional point 6001 and the point 604 being equal to 2 volume units.
• the greatest depth difference d max (that is less or equal to Th2) is selected among ail depth differences de- 2 to deig of the block of pixels 6B and only the adjacent pixel 614 corresponding to the greatest depth difference dmax among all adjacent pixel 612 to 619 is considered with the current pixel 61 1 to generated additional points/cubes (from the corresponding points/cubes 601 and 604).
• the apparatus 9 comprises following elements that are linked together by a data and address bus 91 :
• a power supply e.g. a battery.
• a local memory e.g. a video memory or a RAM (or
• the point cloud 103 may be represented in a picture or in one or more groups of temporally successive pictures, each picture comprising a representation of the point cloud at a determined time 't ⁇
• the one or more groups of temporally successive pictures may form a video representative of at least a part of the point cloud 103,
• the encoded data of the picture 20 is decoded by a decoder DEC1 .
• the decoder DEC1 is compliant with the encoder ENC1 , for example compliant with a legacy decoder such as:
• AVC also named MPEG-4 AVC or h264
• the attributes, encoded at operation 120, are decoded and retrieved, at operation 121 , for example stored in a buffer memory, for use in the generation of a reference picture 125 associated with the picture 20.
• a reference picture 135 (that may be identical to the reference picture 125 of figure 12) may be obtained from the picture by fusing the decoded first attributes obtained from the operation 121 with the second attributes obtained from the operation 123.
• the reference picture may comprise the same structure than the picture, i.e. the same spatial arrangement of the set of images but with different data, i.e. with the decoded first attributes and the obtained second attributes.
• implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted.
• the information may include, for example, instructions for performing a method, or data produced by one of the described implementations.
• a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment.
• Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal.
• the formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream.
• the information that the signal carries may be, for example, analog or digital information.
• the signal may be transmitted over a variety of different wired or wireless links, as is known.
• the signal may be stored on a processor-readable medium.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Graphics (AREA)
• Geometry (AREA)
• Software Systems (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Computing Systems (AREA)
• Image Generation (AREA)
• Processing Or Creating Images (AREA)
• Length Measuring Devices By Optical Means (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)

Priority Applications (8)

Applications Claiming Priority (2)

Publications (1)

Family

ID=60143653

Family Applications (1)

Country Status (9)

Families Citing this family (12)

Citations (1)

Family Cites Families (34)

• 2017
  • 2017-10-06 EP EP17306345.4A patent/EP3467782A1/en not_active Withdrawn
• 2018
  • 2018-10-03 CN CN201880076258.6A patent/CN111386556B/zh active Active
  • 2018-10-03 KR KR1020207012438A patent/KR102537420B1/ko active Active
  • 2018-10-03 JP JP2020518785A patent/JP7407703B2/ja active Active
  • 2018-10-03 HU HUE18786924A patent/HUE061036T2/hu unknown
  • 2018-10-03 US US16/753,787 patent/US11830210B2/en active Active
  • 2018-10-03 WO PCT/US2018/054057 patent/WO2019070778A1/en not_active Ceased
  • 2018-10-03 BR BR112020006530-7A patent/BR112020006530A2/pt unknown
  • 2018-10-03 EP EP18786924.3A patent/EP3692509B1/en active Active
  • 2018-10-03 DK DK18786924.3T patent/DK3692509T3/da active

Patent Citations (1)

Non-Patent Citations (7)

Also Published As

Similar Documents

Legal Events