US20160071274A1 - Selective 3d registration - Google Patents

Selective 3d registration Download PDF

Info

Publication number
US20160071274A1
US20160071274A1 US14/786,977 US201414786977A US2016071274A1 US 20160071274 A1 US20160071274 A1 US 20160071274A1 US 201414786977 A US201414786977 A US 201414786977A US 2016071274 A1 US2016071274 A1 US 2016071274A1
Authority
US
United States
Prior art keywords
entities
entity
parameters
obtaining
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/786,977
Other languages
English (en)
Inventor
Vadim Kosoy
Dani Daniel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MANTISVISION Ltd
Original Assignee
MANTISVISION Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MANTISVISION Ltd filed Critical MANTISVISION Ltd
Priority to US14/786,977 priority Critical patent/US20160071274A1/en
Assigned to MANTISVISION LTD. reassignment MANTISVISION LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DANIEL, Dani, KOSOY, VADIM
Publication of US20160071274A1 publication Critical patent/US20160071274A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G06T7/0024—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • H04N13/0275—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the invention relates to 3D processing and to 3D registration.
  • 3D registration involves an attempt to align two or more 3D models, by finding or applying spatial transformations over the 3D models.
  • 3D registration is useful in many imaging, graphical, image processing, computer vision, medical imaging, robotics, and pattern matching applications.
  • Examples of scenarios were 3D registration involves significant challenges include: a moving 3D camera, or multiple 3D cameras with different positions and directions and generating a plurality of 3D models of a static scene from different viewpoints.
  • the 3D registration process may involve recovering the relative positions and directions of the different viewpoints. Recovering the relative positions and directions of the different viewpoints can further enable merging of the plurality of 3D models into a single high quality 3D model of the scene.
  • the recovered positions and directions can be used in a calibration process of a multiple 3D camera system, or to reconstruct the trajectory of a single moving camera.
  • 3D registration can present some challenges is where a static 3D camera is used to generate a series of 3D models of a moving object or scene.
  • the 3D registration process recovers the relative positions and orientations of the object or scene in each 3D model. Recovering the relative positions and orientations of the object or scene in each 3D model can further enables merging of the plurality of 3D models into a single high quality 3D model of the object or scene. Alternatively, the trajectory of the moving object or scene can be reconstructed.
  • a moving 3D camera or multiple moving 3D cameras, capturing 3D images of a scene that may include several moving objects.
  • the 3D registration results can be used to assemble a map or a model of the environment, for example as input to motion segmentation algorithms, and so forth.
  • the goal of the 3D registration process is to find a spatial transformation between the two models.
  • This can include rigid and non-rigid transformations.
  • the two 3D models may include coinciding parts that correspond to the same objects in the real world, and parts that do not coincide, corresponding to objects (or parts of objects) in the real world that are modeled in only one of the 3D models. Removing the non-coinciding parts speeds up the convergence of the 3D registration process, and can improve the 3D registration result. This same principal extends naturally to the case of three or more 3D models.
  • the 3D registration may be instable due to the geometry of the 3D models that allows two 3D models to “slide” against each other in regions which do not contain enough information to fully constrain the registration, for example, due to uniformity in the appearance of a surface in a certain direction.
  • selecting, or increasing the weights of, the parts of the 3D models that do constrain the registration in the unconstrained direction allows these parts to control the convergence of 3D registration algorithm, may also speed up the convergence of the 3D registration algorithm, and may improve the 3D registration result.
  • a computer implementing a method that include: Given a 3D model that is composed out of n separated entities, a set of parameters is obtained for each entity. A weight can be calculated for each entity, giving higher weight for entities corresponding to rarer parameters. Entities can be assigned to components based on their corresponding parameters. Entities can sample based on the weights or based on the components to obtain a subset of all entities. A new 3D model is constructed from the subset of sampled entities, thereby producing a smaller 3D model. Computations, such as 3D registration, can be performed using the smaller 3D model. This can speed-up the execution of 3D algorithms, while the sampling that preserves entities with rare parameters maintain entities that may have special importance in the execution of the algorithm.
  • FIG. 1 is a simplified block diagram of an example for one possible implementation of a mobile communication device with 3D capturing capabilities.
  • FIG. 2 is a simplified block diagram of an example for one possible implementation of a system that includes a mobile communication device with 3D capturing capabilities and a cloud platform.
  • FIG. 3 is an illustration of a possible scenario in which a plurality of 3D models is generated by a single 3D camera.
  • FIG. 4 is an illustration of a possible scenario in which a plurality of 3D models is generated by a plurality of 3D cameras.
  • ⁇ should be expansively construed to cover any kind of electronic device, component or unit with data processing capabilities, including, by way of non-limiting example, a personal computer, a tablet, a smartphone, a server, a computing system, a communication device, a processor (for example, digital signal processor (DSP), and possibly with embedded memory), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), and so on), a core within a processor, any other electronic computing device, and or any combination thereof.
  • DSP digital signal processor
  • FPGA field programmable gate array
  • ASIC application specific integrated circuit
  • GPU graphics processing unit
  • the phrase “for example,” “such as”, “for instance” and variants thereof describe non-limiting embodiments of the presently disclosed subject matter.
  • Reference in the specification to “one case”, “some cases”, “other cases” or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment(s) is included in at least one embodiment of the presently disclosed subject matter.
  • the appearance of the phrase “one case”, “some cases”, “other cases” or variants thereof does not necessarily refer to the same embodiment(s).
  • one or more stages illustrated in the figures may be executed in a different order and/or one or more groups of stages may be executed simultaneously and vice versa.
  • the figures illustrate a general schematic of the system architecture in accordance with an embodiment of the presently disclosed subject matter.
  • Each module in the figures can be made up of any combination of software, hardware and/or firmware that performs the functions as defined and explained herein.
  • the modules in the figures may be centralized in one location or dispersed over more than one location.
  • 3D model is recognized by those with ordinary skill in the art and refers to any kind of representation of any 3D surface, 3D object, 3D scene, 3D prototype, 3D shape, 3D design and so forth, either static or moving.
  • a 3D model can be represented in a computer in different ways. Some example includes the popular range image, where one associate a depth for pixels of a regular 2D image. Another simple example is the point cloud, where the model consists of a set of 3D points. A different example is using polygons, where the model consists of a set of polygons.
  • Special types of polygon based models include: (i) polygon soup, where the polygons are unsorted; (ii) mesh, where the polygons are connected to create a continuous surface; (iii) subdivision surface, where a sequence of meshes is used to approximate a smooth surface; (iv) parametric surface, where a set of formulas are used to describe a surface; (v) implicit surface, where one or more equations are used to describe a surface; (vi) and so forth. Another example is to represent a 3D model as a skeleton model, where a graph of curves with radii is used. Additional examples include a mixture of any of the above methods. There are also many variants on the above methods, as well as a variety of other methods. It is important to note that one may convert one kind of representation to another, at the risk of losing some information, or by making some assumptions to complete missing information.
  • 3D registration process is recognized by those with ordinary skill in the art and refers to the process of finding one or more spatial transformations that aligns two or more 3D models, and/or for transforming two or more 3D models into a single coordinate system.
  • 3D registration algorithm is recognized by those with ordinary skill in the art and refers to any process, algorithm, method, procedure, and/or technique, for solving and/or approximating one or more solutions to the 3D registration process.
  • Some examples for 3D registration algorithms include the Iterative Closest Point algorithm, the Robust Point Matching algorithm, the Kernel Correlation algorithm, the Coherent Point Drift algorithm, RANSAC based algorithms, any graph and/or hypergraph matching algorithm, any one of the many variants of these algorithms, and so forth.
  • iterative 3D registration algorithm is recognized by those with ordinary skill in the art and refers to a 3D registration algorithm that repeatedly adjusts an estimation of the 3D registration until convergence, possibly starting from an initial guess for the 3D registration.
  • 3D registration result is recognized by those with ordinary skill in the art and refers to the product of a 3D registration algorithm. This may be in the form of: spatial transformations between pairs of 3D models; spatial transformations for transforming all the 3D models into a single coordinate system; representation of all the 3D models in a single coordinate system; and so forth.
  • estimate 3D registration is recognized by those with ordinary skill in the art and refers to any estimation of 3D registration result.
  • the estimation may be a random guess for the 3D registration result.
  • each iteration updates an estimation of 3D registration result to obtain a new estimation.
  • a 3D registration result by itself can also be an estimated 3D registration. And so forth.
  • 3D camera is recognized by those with ordinary skill in the art and refers to any type of device, including a camera and/or a sensor, which is capable of capturing 3D images, 3D videos, and/or 3D models. Examples include: stereoscopic cameras, time-of-flight cameras, obstructed light sensors, structured light sensors, and so forth.
  • FIG. 1 is a simplified block diagram of an example for one possible implementation of a mobile communication device with 3D capturing capabilities.
  • the mobile communication device 100 can includes a 3D camera 10 that is capable of providing 3D depth or range data.
  • a 3D camera 10 that is capable of providing 3D depth or range data.
  • FIG. 1 there is shown a configuration of an active stereo 3D camera, but in further examples of the presently disclosed subject matter other known 3D cameras can be used.
  • Those versed in the art can readily apply the teachings provided in the examples of the presently disclosed subject matter to other 3D camera configurations and to other 3D capture technologies.
  • the 3D camera 10 can include: a 3D capture sensor 12 , a driver 14 , a 3D capture processor 16 and a flash module 18 .
  • the flash module 18 is configured to project a structured light pattern and the 3D capture sensor 12 is configured to capture an image which corresponds to the reflected pattern, as reflected from the environment onto which the pattern was projected.
  • U.S. Pat. No. 8,090,194 to Gordon et. al. describes an example structured light pattern that can be used in a flash component of a 3D camera, as well as other aspects of active stereo 3D capture technology and is hereby incorporated into the present application in its entirety.
  • International Application Publication No. WO2013/144952 describes an example of a possible flash design and is hereby incorporated by reference in its entirety.
  • the flash module 18 can include an IR light source, such that it is capable of projecting IR radiation or light
  • the 3D capture sensor 12 can be and IR sensor, that is sensitive to radiation in the IR band, and such that it is capable of capturing the IR radiation that is returned from the scene.
  • the flash module 18 and the 3D capture sensor 12 are calibrated.
  • the driver 14 , the 3D capture processor 16 or any other suitable component of the mobile communication device 100 can be configured to implement auto-calibration for maintaining the calibration among the flash module 18 and the 3D capture sensor 12 .
  • the 3D capture processor 16 can be configured to perform various processing functions, and to run computer program code which is related to the operation of one or more components of the 3D camera.
  • the 3D capture processor 16 can include memory 17 which is capable of storing the computer program instructions that are executed or which are to be executed by the processor 16 .
  • the driver 14 can be configured to implement a computer program which operates or controls certain functions, features or operations that the components of the 3D camera 10 are capable of carrying out.
  • the mobile communication device 100 can also include hardware components in addition to the 3D camera 10 , including for example, a power source 20 , storage 30 , a communication module 40 , a device processor 50 and memory 60 , device imaging hardware 110 , a display unit 120 , and other user interfaces 130 .
  • one or more components of the mobile communication device 100 can be implemented as distributed components.
  • a certain component can include two or more units distributed across two or more interconnected nodes.
  • a computer program possibly executed by the device processor 50 , can be capable of controlling the distributed component and can be capable of operating the resources on each of the two or more interconnected nodes.
  • the power source 20 can include one or more power source units, such as a battery, a short-term high current source (such as a capacitor), a trickle-charger, etc.
  • the device processor 50 can include one or more processing modules which are capable of processing software programs.
  • the processing module can each have one or more processors.
  • the device processor 50 may include different types of processors which are implemented in the mobile communication device 100 , such as a main processor, an application processor, etc.
  • the device processor 50 or any of the processors which are generally referred to herein as being included in the device processor can have one or more cores, internal memory or a cache unit.
  • the storage unit 30 can be configured to store computer program code that is necessary for carrying out the operations or functions of the mobile communication device 100 and any of its components.
  • the storage unit 30 can also be configured to store one or more applications, including 3D applications 80 , which can be executed on the mobile communication device 100 .
  • 3D applications 80 can be stored on a remote computerized device, and can be consumed by the mobile communication device 100 as a service.
  • the storage unit 30 can be configured to store data, including for example 3D data that is provided by the 3D camera 10 .
  • the communication module 40 can be configured to enable data communication to and from the mobile communication device.
  • examples of communication protocols which can be supported by the communication module 40 include, but are not limited to cellular communication (3G, 4G, etc.), wired communication protocols (such as Local Area Networking (LAN)), and wireless communication protocols, such as Wi-Fi, wireless personal area networking (PAN) such as Bluetooth, etc.
  • the components of the 3D camera 10 can be implemented on the mobile communication hardware resources.
  • the device processor 50 can be used instead of having a dedicated 3D capture processor 16 .
  • the mobile communication device 100 can include more than one processor and more than one type of processor, e.g., one or more digital signal processors (DSP), one or more graphical processing units (GPU), etc., and the 3D camera can be configured to use a specific one (or a specific set or type) processor(s) from the plurality of device 100 processors.
  • DSP digital signal processors
  • GPU graphical processing units
  • the mobile communication device 100 can be configured to run an operating system 70 .
  • mobile device operating systems include but are not limited to: such as Windows MobileTM by Microsoft Corporation of Redmond, Wash., and the Android operating system developed by Google Inc. of Mountain View, Calif.
  • the 3D application 80 can be any application which uses 3D data. Examples of 3D applications include a virtual tape measure, 3D video, 3D snapshot, 3D modeling, etc. It would be appreciated that different 3D applications can have different requirements and features.
  • a 3D application 80 may be assigned to or can be associated with a 3D application group.
  • the device 100 can be capable of running a plurality of 3D applications 80 in parallel.
  • Imaging hardware 110 can include any imaging sensor, in a particular example, an imaging sensor that is capable of capturing visible light images can be used.
  • the imaging hardware 110 can include a sensor, typically a sensor that is sensitive at least to visible light, and possibly also a light source (such as one or more LEDs) for enabling image capture in low visible light conditions.
  • the device imaging hardware 110 or some components thereof can be calibrated to the 3D camera 10 , and in particular to the 3D capture sensor 12 and to the flash 18 . It would be appreciated that such a calibration can enable texturing of the 3D image and various other co-processing operations as will be known to those versed in the art.
  • the imaging hardware 110 can include a RGB-IR sensor that is used for capturing visible light images and for capturing IR images.
  • the RGB-IR sensor can serve as the 3D capture sensor 12 and as the visible light camera.
  • the driver 14 and the flash 18 of the 3D camera, and possibly other components of the device 100 are configured to operate in cooperation with the imaging hardware 110 , and in the example given above, with the RGB-IR sensor, to provide the 3D depth or range data.
  • the display unit 120 can be configured to provide images and graphical data, including a visual rendering of 3D data that was captured by the 3D camera 10 , possibly after being processed using the 3D application 80 .
  • the user interfaces 130 can include various components which enable the user to interact with the mobile communication device 100 , such as speakers, buttons, microphones, etc.
  • the display unit 120 can be a touch sensitive display which also serves as a user interface.
  • any processing unit including the 3D capture processor 16 or the device processor 50 and/or any sub-components or CPU cores, etc. of the 3D capture processor 16 and/or the device processor 50 , can be configured to read 3D images and/or frames of 3D video clips stored in storage unit 30 , and/or to receive 3D images and/or frames of 3D video clips from an external source, for example through communication module 40 ; produce 3D models out of said 3D images and/or frames.
  • the produced 3D models can be stored in storage unit 30 , and/or sent to an external destination through communication module 40 .
  • any such processing unit can be configured to execute 3D registration on a plurality of 3D models.
  • FIG. 2 is a simplified block diagram of an example for one possible implementation of a system 200 , that includes a mobile communication device with 3D capturing capabilities 100 , and a cloud platform 210 .
  • the cloud platform 210 can include hardware components, including for example, one or more power sources 220 , one or more storage units 230 , one or more communication modules 240 , one or more processors 250 , optionally one or more memory units 260 , and so forth.
  • the storage unit 230 can be configured to store computer program code that is necessary for carrying out the operations or functions of the cloud platform 210 and any of its components.
  • the storage unit 230 can also be configured to store one or more applications, including 3D applications, which can be executed on the cloud platform 210 .
  • the storage unit 230 can be configured to store data, including for example 3D data.
  • the communication module 240 can be configured to enable data communication to and from the cloud platform.
  • examples of communication protocols which can be supported by the communication module 240 include, but are not limited to cellular communication (3G, 4G, etc.), wired communication protocols (such as Local Area Networking (LAN)), and wireless communication protocols, such as Wi-Fi, wireless personal area networking (PAN) such as Bluetooth, etc.
  • the one or more processors 250 can include one or more processing modules which are capable of processing software programs.
  • the processing module can each have one or more processing units.
  • the device processor 250 may include different types of processors which are implemented in the cloud platform 210 , such as general purpose processing units, graphic processing units, physics processing units, etc.
  • the device processor 250 or any of the processors which are generally referred to herein can have one or more cores, internal memory or a cache unit.
  • the one or more memory units 260 may include several memory units. Each unit may be accessible by all of the one or more processors 250 , or only by a subset of the one or more processors 250 .
  • any processing unit including the one or more processors 250 and/or any sub-components or CPU cores, etc. of the one or more processors 250 , can be configured to read 3D images and/or frames of 3D video clips stored in storage unit 230 , and/or to receive 3D images and/or frames of 3D video clips from an external source, for example through communication module 240 , where, by a way of example, the communication module may be communicating with the mobile communication device 100 , with another cloud platform, and so forth.
  • the processing unit can be further configured to produce 3D models out of said 3D images and/or frames.
  • the produced 3D models can be stored in storage unit 230 , and/or sent to an external destination through communication module 240 .
  • any such processing unit can be configured to execute 3D registration on a plurality of 3D models.
  • FIG. 3 is an illustration of a possible scenario in which a plurality of 3D models is generated by a single 3D camera.
  • a moving object is captured at two sequential points in time. Denote the earliest point in time as T1, and the later point in time as T2.
  • 311 is the object at T1
  • 312 is the object at T2.
  • 321 is the single 3D camera at time T1, which generates a 3D model 331 of the object at time T1 ( 311 ).
  • the single 3D camera ( 322 ) generates the 3D model 332 of the object ( 312 ).
  • 3D registration is used to align 3D model 331 with 3D model 332 . Further by a way of example, the 3D registration result can be used to reconstruct the trajectory of the moving object 311 and 312 .
  • FIG. 4 is an illustration of a possible scenario in which a plurality of 3D models is generated by a plurality of 3D cameras.
  • a single object 410 is captured by two 3D cameras: 3D camera 421 generates the 3D model 431 , and 3D camera 422 generates the 3D model 432 .
  • 3D registration is used to align 3D model 431 with 3D model 432 .
  • the 3D registration result can be used to reconstruct a single combined 3D model of the object 410 from the two 3D models 431 and 432 .
  • a 3D model consists of a group of separated entities, possibly while holding additional information about the relations among the entities.
  • an entity when representing the 3D model as a point cloud, an entity can be a point; when representing the 3D model as a group of polygons, the entity may be a polygon; when representing the 3D model as a skeleton model, each curve and/or a radii may be an entity; when representing the 3D model as a graph or a hypergraph, each node and/or vertex may be an entity; and so forth.
  • a set of parameters related to that entity are obtained.
  • the parameters may include: the normal to the polygon, the surface area of the polygon, the circumference of the polygon, and so forth.
  • the parameters may include: the normal associated with the point entity in the point cloud, the distance between the point entity and the nearest other point in the point cloud, the density of points around the point entity in the point cloud, and so on.
  • the parameters may include: the length of the radius entity, the orientation of the radii entity, the size of the adjacent curves, and so forth.
  • other examples of parameters related to the entity include: accuracy estimation provided by the 3D model capturing process for this entity, the color histogram related to the entity, parameters extracted from a 2D image of a patch of a 2D image related to that entity, and so forth.
  • parameters related to an entity when a 3D registration result or an estimated 3D registration is available, the parameters related to an entity may be extracted from the neighborhood or region of the second 3D model that the entity is nearest to.
  • parameters related to an entity in the case of an iterative 3D registration algorithm, may include information regarding the progress of the iterative 3D registration algorithm with regard to that entity.
  • the entity neighborhood may be the entire group of separated entities.
  • the entity neighborhood may be a subset containing only the entity itself.
  • the entity neighborhood may include all entities within a specified radius from the entity.
  • the entity neighborhood may be an empty subset. And so forth.
  • each entity a set containing all the sets of parameters relating to entities in the entity neighborhood is calculated, which will be referred to hereafter as the entity parameters' neighborhood.
  • the entity parameters' neighborhood is a set of parameters, and that for each entity in the entity neighborhood there is a corresponding entry in the entity parameters' neighborhood and vice versa. Therefore, the number of entities in the entity neighborhood equals to the number of sets of parameters in the entity parameters' neighborhood.
  • the entity's crowdedness can be calculated as a measure related to the density of and/or around the parameters related to that entity, with respect to the entity parameters' neighborhood. Further by a way of example, a similarity and/or distance between the entity parameters and any other set of parameters in the entity parameters' neighborhood can be calculated. The density can be calculated as a function of these similarities and/or distances.
  • each set of parameters translates into a point in the parameters' space. Since each entity corresponds to a set of parameters, each entity also corresponds to a point in the parameters' space, which will be referred to hereafter as the entity's point. Further by a way of example, the density related to an entity can be calculated as the density of the entity's point, or a function of the entity's point density.
  • an entity parameters' neighborhood corresponds to a group of points in the parameters' space, which will be referred to hereafter as the entity's neighborhood points.
  • the density related to an entity can be calculated as the density of the entity's point with respect to the entity's neighborhood points, or a function of the entity's point density with respect to the entity's neighborhood points.
  • the distance from the entity's point to any other point in the entity's neighborhood points is calculated, thereby ending with one distance for each point in the entity's neighborhood points.
  • the density can be calculated as a function of these distances.
  • the number of distances lower than a certain threshold can be counted, and this count can serve as a measure of density.
  • a count is made for each threshold by counting the number of distances lower than this threshold, thereby producing n counts.
  • the density can be calculated as a function of these counts. Further by a way of example, given two thresholds, the density can be calculated as the ratio between the two counts corresponding to the two thresholds.
  • the density can be set to be a function of these similarities, for example, a sum of these similarities.
  • each entity of the 3D model's group of separated entities can be assigned to one or more components based the entity's parameters, based on one or more assignment rules that are based on the set of parameters that relate to the entity . Therefore, each component is a subset of the 3D model's group of separated entities. Further by a way of example, each component corresponds to a component size, for example the component size is the number of entities assigned to that component. Further by a way of example, an entity's crowdedness can be calculated as the number of components the entity is assigned to. Further by a way of example, an entity's crowdedness can be calculated as a function of the component sizes of all the components which the entity is assigned to. Further by a way of example, the entity's crowdedness can be set to any statistical function of these component sizes, including the average size, the median size, the maximal size, the minimal size, and so forth.
  • each entity may be assigned to at most one component, meaning that the components are disjoint.
  • an entity's crowdedness can be calculated as a function of the size of the single component the entity is assigned to.
  • an entity's crowdedness can be calculated as an indicator that is 1 in case the entity is assigned to a component, and 0 otherwise.
  • examples for assignment rule for assigning of the entities to components includes: when the parameters related to an entity are a normal vector, the assignment rule can be based on the normal vector orientation. For example, if the normal vector is represented in spherical coordinates as, ( ⁇ , ⁇ ), and given two positive integers, n and m, we can define n ⁇ m components using the following rules,
  • 0 ⁇ k ⁇ n, and, 0 ⁇ 1 ⁇ m are indices of the components.
  • an entity is assigned to a component if the ray that starts from the axes origin in the direction of the normal intersect with the face, and in case the normal interests with an edge, a corner or a vertex of the polyhedron, the entity is assigned to all components corresponding to faces adjacent to the intersected edge, corner or vertex.
  • each component may correspond to a spectrum of colors, and the assignment rule can be to assign an entity for each component that correspond to a spectrum that include that color.
  • a weight for the entity is calculated, which will be referred to hereafter as the entity's weight.
  • the entity's weight can be calculated as, 1/r, as, exp( ⁇ r/a), where a is a scaling factor, and so forth.
  • a 3D registration algorithm is invoked on a plurality of 3D models, where at least one 3D model of the plurality of 3D models consists of a group of separated entities, possibly while holding additional information about the relations among the entities, and where the at least one 3D model is accompanied by entity's weight for all or some entities of the at least one 3D model.
  • a sampling of the entities that is based on the entity's weight can be perform, thereby producing a new 3D model that consists only from the sample of the entity.
  • the sample may be random, where the probability for sampling an entity is proportional to the entity's weight.
  • a 3D registration algorithm is invoked on the new 3D model that consist only from the sample of the entity, and on one or more 3D models.
  • a sampling of the entities that is based on the components can be performed, thereby producing a new 3D model that consists only from the sample of the entity.
  • the sample may be random, selecting a subset of entities from each component, where the number of entities sampled from each component is controlled by the size of the component.
  • a 3D registration algorithm is invoked on the new 3D model that consist only from the sample of the entity, and on one or more 3D models.
  • the above scheme can also be applied to 2D model. Assuming that the 2D model is constructed out of entities, a set of parameters can be calculated for each entity; a density may be calculated for each set of parameters; a crowdedness value can be calculated for each entity; entities can be assigned to components; weights can be calculated for the different entities; sampling can be performed based on these weights; and so forth.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Architecture (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
US14/786,977 2013-04-30 2014-04-30 Selective 3d registration Abandoned US20160071274A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/786,977 US20160071274A1 (en) 2013-04-30 2014-04-30 Selective 3d registration

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201361817496P 2013-04-30 2013-04-30
US201361817502P 2013-04-30 2013-04-30
US14/786,977 US20160071274A1 (en) 2013-04-30 2014-04-30 Selective 3d registration
PCT/IL2014/050391 WO2014178051A2 (fr) 2013-04-30 2014-04-30 Enregistrement 3d sélectif

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2014/050391 A-371-Of-International WO2014178051A2 (fr) 2013-04-30 2014-04-30 Enregistrement 3d sélectif

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/822,157 Continuation US10861174B2 (en) 2013-04-30 2017-11-26 Selective 3D registration

Publications (1)

Publication Number Publication Date
US20160071274A1 true US20160071274A1 (en) 2016-03-10

Family

ID=51844051

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/786,977 Abandoned US20160071274A1 (en) 2013-04-30 2014-04-30 Selective 3d registration
US15/822,157 Active US10861174B2 (en) 2013-04-30 2017-11-26 Selective 3D registration

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/822,157 Active US10861174B2 (en) 2013-04-30 2017-11-26 Selective 3D registration

Country Status (2)

Country Link
US (2) US20160071274A1 (fr)
WO (1) WO2014178051A2 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160071274A1 (en) 2013-04-30 2016-03-10 Mantisvision Ltd. Selective 3d registration
CN112634181A (zh) * 2019-09-24 2021-04-09 北京百度网讯科技有限公司 用于检测地面点云点的方法和装置

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120177283A1 (en) * 2011-01-11 2012-07-12 Sen Wang Forming 3d models using two images
US20130244782A1 (en) * 2011-01-31 2013-09-19 Microsoft Corporation Real-time camera tracking using depth maps

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7139409B2 (en) * 2000-09-06 2006-11-21 Siemens Corporate Research, Inc. Real-time crowd density estimation from video
US7215810B2 (en) * 2003-07-23 2007-05-08 Orametrix, Inc. Method for creating single 3D surface model from a point cloud
DE602006021627D1 (de) * 2005-12-16 2011-06-09 Technion Res And Dev Of Foundation Ltd Verfahren und vorrichtung zur bestimmung der ähnlichkeit zwischen oberflächen
US8090194B2 (en) 2006-11-21 2012-01-03 Mantis Vision Ltd. 3D geometric modeling and motion capture using both single and dual imaging
US8538166B2 (en) 2006-11-21 2013-09-17 Mantisvision Ltd. 3D geometric modeling and 3D video content creation
US8675951B2 (en) * 2007-05-11 2014-03-18 Three Pixels Wide Pty Ltd. Method and system for generating a 3D model
WO2009003225A1 (fr) * 2007-06-29 2009-01-08 Adelaide Research & Innovation Pty Ltd Procédé et système de génération d'un modèle tridimensionnel à partir d'images
CN102918846B (zh) * 2010-02-24 2015-09-09 日本电信电话株式会社 多视点视频编码方法、多视点视频解码方法、多视点视频编码装置、多视点视频解码装置
US9404986B2 (en) * 2011-05-06 2016-08-02 The Regents Of The University Of California Measuring biological tissue parameters using diffusion magnetic resonance imaging
US9053571B2 (en) * 2011-06-06 2015-06-09 Microsoft Corporation Generating computer models of 3D objects
EP3239652B1 (fr) 2012-03-26 2019-10-30 Mantisvision Ltd. Appareil photo tridimensionnel et projecteur associé
US20160071274A1 (en) 2013-04-30 2016-03-10 Mantisvision Ltd. Selective 3d registration
US9613388B2 (en) * 2014-01-24 2017-04-04 Here Global B.V. Methods, apparatuses and computer program products for three dimensional segmentation and textured modeling of photogrammetry surface meshes

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120177283A1 (en) * 2011-01-11 2012-07-12 Sen Wang Forming 3d models using two images
US20130244782A1 (en) * 2011-01-31 2013-09-19 Microsoft Corporation Real-time camera tracking using depth maps

Also Published As

Publication number Publication date
WO2014178051A3 (fr) 2014-12-24
WO2014178051A2 (fr) 2014-11-06
US20180330514A1 (en) 2018-11-15
US10861174B2 (en) 2020-12-08

Similar Documents

Publication Publication Date Title
US11238606B2 (en) Method and system for performing simultaneous localization and mapping using convolutional image transformation
US10796482B2 (en) 3D hand shape and pose estimation
CN111243093B (zh) 三维人脸网格的生成方法、装置、设备及存储介质
CN111598998B (zh) 三维虚拟模型重建方法、装置、计算机设备和存储介质
CN109074660B (zh) 单目相机实时三维捕获和即时反馈的方法和系统
US9251590B2 (en) Camera pose estimation for 3D reconstruction
WO2020206903A1 (fr) Procédé et dispositif de mise en correspondance d'images et support de mémoire lisible par ordinateur
CN109683699B (zh) 基于深度学习实现增强现实的方法、装置及移动终端
US9922447B2 (en) 3D registration of a plurality of 3D models
US20180189556A1 (en) Hand gesture recognition for virtual reality and augmented reality devices
CN111062981B (zh) 图像处理方法、装置及存储介质
CN109242961A (zh) 一种脸部建模方法、装置、电子设备和计算机可读介质
US20230169677A1 (en) Pose Estimation Method and Apparatus
CN111357034A (zh) 点云生成方法、系统和计算机存储介质
CN111161398B (zh) 一种图像生成方法、装置、设备及存储介质
US20160189339A1 (en) Adaptive 3d registration
US20230140170A1 (en) System and method for depth and scene reconstruction for augmented reality or extended reality devices
US10861174B2 (en) Selective 3D registration
CN116109799B (zh) 调整模型训练方法、装置、计算机设备及存储介质
Anasosalu et al. Compact and accurate 3-D face modeling using an RGB-D camera: let's open the door to 3-D video conference
CN116704029A (zh) 稠密物体语义地图构建方法、装置、存储介质及电子设备
US11908096B2 (en) Stereoscopic image acquisition method, electronic device and storage medium
CN115760888A (zh) 图像处理方法、装置、计算机及可读存储介质
RU2757563C1 (ru) Способ визуализации 3d портрета человека с измененным освещением и вычислительное устройство для него
WO2024064419A1 (fr) Procédé et dispositif de reconstruction tridimensionnelle, et support de stockage

Legal Events

Date Code Title Description
AS Assignment

Owner name: MANTISVISION LTD., ISRAEL

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KOSOY, VADIM;DANIEL, DANI;SIGNING DATES FROM 20151116 TO 20151122;REEL/FRAME:037137/0220

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION