US20030063086A1 - 3D computer model processing apparatus - Google Patents

3D computer model processing apparatus Download PDF

Info

Publication number
US20030063086A1
US20030063086A1 US10255178 US25517802A US2003063086A1 US 20030063086 A1 US20030063086 A1 US 20030063086A1 US 10255178 US10255178 US 10255178 US 25517802 A US25517802 A US 25517802A US 2003063086 A1 US2003063086 A1 US 2003063086A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
silhouette
polygon mesh
mesh
reference
projection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10255178
Other versions
US7079680B2 (en )
Inventor
Adam Baumberg
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Europa NV
Original Assignee
Canon Europa NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205Re-meshing
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

In a computer processing apparatus 2, a 3D computer model comprising a polygon mesh 500 representing the visual hull of an object 300 is generated by processing images of the object recorded at different positions and orientations to back-project the silhouette of the object in each image to give a respective cone which constrains the volume of 3D space occupied by the object. To remove concave and convex artefacts 510 in the polygon mesh 500, the polygon mesh is projected into each image to give a respective reference silhouette for each image. A change is made to at least one edge or vertex in the polygon mesh to give a refined polygon mesh, which is then projected into each image. The resulting silhouette in each image is tested against the corresponding reference silhouette. If the silhouette of the refined polygon mesh is the same as each reference silhouette, to within an allowable tolerance, then the changed edge or vertex is allowed in the polygon mesh, otherwise the polygon mesh is returned to its previous state. This processing to change and test the polygon mesh is carried out a number of times. Artefacts are removed from the polygon mesh because their removal does not make the polygon mesh inconsistent with the reference silhouettes. Parts of the polygon mesh representing actual features of the object are not changed because such a change causes the polygon mesh to become inconsistent with at least one reference silhouette and therefore such a change is rejected.

Description

  • The present invention relates to computer processing of data defining a three-dimensional (3D) computer model of an object. [0001]
  • Many methods are known for generating a 3D computer model of an object. [0002]
  • The known methods include “shape-from-silhouette” methods, which generate a 3D computer model by processing images of an object recorded at known positions and orientations to back project the silhouette of the object in each image to give a respective endless cone containing the object and having its appex at the position of the focal point of the camera when the image was recorded. Each cone therefore constrains the volume of 3D space occupied by the object, and the intersection of the cones, which approximates the object and is known as the “visual hull” of the object, is calculated. [0003]
  • Examples of shape-from-silhouette methods are described, for example, in “Looking to build a model world: automatic construction of static object models using computer vision” by Illingworth and Hilton in Electronics and Communication Engineering Journal, June 1998, pages 103-113, and “Automatic reconstruction of 3D objects using a mobile camera” by Niem in Image and Vision Computing 17 (1999) pages 125-134. The methods described in both of these papers calculate the intersections of the silhouette cones to generate a “volume representation” of the object made up of a plurality of voxels (cuboids). More particularly, 3D space is divided into voxels, and the voxels are tested to determine which ones lie inside the volume defined by the intersection of the silhouette cones. Voxels inside the intersection volume are retained to define a volume of voxels representing the object. The volume representation is then converted to a surface model comprising a plurality of polygons for rendering. [0004]
  • “A Volumetric Intersection Algorithm for 3d-Reconstruction Using a Boundary-Representation” by Martin Löhlein at http://i31www.ira.uka.de/diplomarbeiten/da_martin_loehlein/Reconstruction.html discloses a shape-from-silhouette method of generating a 3D computer model which does not result in a voxel representation. Instead, the intersections of the silhouette cones from a plurality of images are calculated directly. More particularly, the method starts with a cube containing the object, and intersects it with the first silhouette cone to give a first approximation of the object. This approximation is then intersected with the next cone to give a second approximation, and so on for each respective silhouette cone. To intersect a silhouette cone with an approximation, the cone and the approximation are projected into the image from which the cone was taken. This reduces the cone to the 2d-polygon (silhouette) from which it was made and reduces the approximation from 3d-polygons to 2d-polygons. The cone polygon is then intersected with all the approximation's polygons using a conventional algorithm for 2d-polygon intersection. [0005]
  • The inventor of the present invention has realised that a visual hull computer model of an object often suffers from the problem that artefacts comprising concavities and convexities appear in the computer world which do not exist on the object in real-life. [0006]
  • In particular, the inventor has identified two types of convex and concave artefacts which decrease the accuracy of the 3D computer model of an object and which are a particular problem. These are convex artefacts on top of planar surfaces forming a “dome” on the planar surface, and convex and concave artefacts in high curvature surface regions forming “creases” and “folds” in the surface that are not present on the object. [0007]
  • The present invention has been made with these problems in mind. [0008]
  • According to the present invention, there is provided a computer processing method and apparatus for processing data defining a polygon mesh representing the visual hull of an object, to remove artefacts in the computer model. At least one polygon in the mesh is changed, and the resulting change to the polygon mesh is tested against a reference silhouette in each image to determine whether it is consistent to within a predetermined tolerance with all of the reference silhouettes. A change which is consistent with all of the reference silhouettes is allowed in the computer model (or a corresponding change producing the same effect is allowed), but a change which is inconsistent with at least one reference silhouette is rejected. The process of making a change and testing it against the reference silhouettes is repeated a number of times. [0009]
  • In this way, artefacts are removed because their removal does note make the polygon mesh inconsistent with the reference silhouettes. On the other hand, parts of the polygon mesh representing features actually present on the object are not changed because such a change causes the polygon mesh to become inconsistent with at least one reference silhouette, and therefore such a change is rejected. [0010]
  • Preferably, the reference silhouettes are generated by projection of the polygon mesh into each images before any changes are made to the polygon mesh. [0011]
  • The present invention also provides a computer program product, embodied for example as a storage device carrying instructions or a signal carrying instructions, comprising instructions for causing a programmable processing apparatus to become operable to perform a method as set out above or to become configured as an apparatus as set out above.[0012]
  • Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: [0013]
  • FIG. 1 schematically shows the components of a first embodiment of the invention, together with the notional functional processing units into which the processing apparatus component may be thought of as being configured when programmed by programming instructions; [0014]
  • FIG. 2 shows the processing operations performed by the processing apparatus in FIG. 1 to process input data; [0015]
  • FIG. 3 shows an example to illustrate images and their recording positions and orientations defined in the input data stored at step S[0016] 2-2 in FIG. 2;
  • FIG. 4 shows an example of polyhedra defined from object silhouettes in the images from FIG. 3, and illustrates the processing performed at step S[0017] 2-8 in FIG. 2 to calculate the intersections of the polyhedra to generate a 3D computer model of the subject object;
  • FIG. 5, comprising FIGS. 5[0018] a and 5 b, shows an example of a polygon mesh generated as a result of the processing at step S2-8 in FIG. 2 and an example of a convex artefact in the polygon mesh;
  • FIG. 6 shows the processing operations performed at step S[0019] 2-10 in the first embodiment to refine the polygon mesh to remove artefacts;
  • FIG. 7, comprising FIGS. 7[0020] a and 7 b, shows the processing operations performed at step S6-10 in FIG. 6 and step S8-10 in FIG. 8 to determine whether a refined polygon mesh is consistent with reference silhouettes; and
  • FIG. 8, comprising FIGS. 8[0021] a and 8 b, shows the processing operations performed at step S2-10 in a second embodiment to refine the polygon mesh to remove artefacts.
  • First Embodiment [0022]
  • Referring to FIG. 1, an embodiment of the invention comprises a processing apparatus [0023] 2, such as a personal computer, containing, in a conventional manner, one or more processors, memories, graphics cards etc, together with a display device 4, such as a conventional personal computer monitor, and user input devices 6, such as a keyboard, mouse etc.
  • The processing apparatus [0024] 2 is programmed to operate in accordance with programming instructions input, for example, as data stored on a data storage medium, such as disk 12, and/or as a signal 14 input to the processing apparatus 2, for example from a remote database, by transmission over a communication network (not shown) such as the Internet or by transmission through the atmosphere, and/or entered by a user via a user input device 6 such as a keyboard.
  • As will be described in more detail below, the programming instructions comprise instructions to cause the processing apparatus [0025] 2 to become configured to process input data defining a plurality of images of a subject object recorded at different positions and orientations and data defining the positions and orientations at which the input images were recorded, to generate data defining a three-dimensional computer model of the subject object and subsequently to refine the model to remove artefacts. The computer model is calculated by back-projecting the silhouette of the subject object in each input image to define a polyhedron for each silhouette which constrains the volume of 3D space occupied by the subject object. The volume of 3D space defined by the intersection of the polyhedra is calculated. This defines the largest object in volume, which, when imaged, could generate the input images, and is referred to as the visual hull of the subject object. The refinement processing is carried out to remove concavities and convexities which the inventor has found are often present as artefacts in the model because the model is a visual hull model.
  • When programmed by the programming instructions, processing apparatus [0026] 2 can be thought of as being configured as a number of functional units for performing processing operations. Examples of such functional units and their interconnections are shown in FIG. 1. The units and interconnections illustrated in FIG. 1 are, however, notional, and are shown for illustration purposes only to assist understanding; they do not necessarily represent units and connections into which the processor, memory etc of the processing apparatus 2 actually become configured.
  • Referring to the functional units shown in FIG. 1, a central controller [0027] 20 is arranged to process inputs from the user input devices 6, and also to provide control and processing for the other functional units. Memory 30 is provided for use by central controller 20 and the other functional units.
  • Input data store [0028] 40 is arranged to store input data input to the processing apparatus 2 for example as data stored on a storage device, such as disk 42, as a signal 44 transmitted to the processing apparatus 2, or using a user input device 6. The input data defines a plurality of images of a subject object recorded at different positions and orientations, and data defining the relative positions and orientations at which the images were recorded. In addition, in this embodiment, the input data also includes data defining the intrinsic parameters of the camera which recorded the images, that is, the aspect ratio, focal length, principal point (the point at which the optical axis intersects the imaging plane), first order radial distortion coefficient, and skew angle (the angle between the axes of the pixel grid; because the axes may not be exactly orthogonal).
  • The input data defining the input images may be generated for example by downloading pixel data from a digital camera which recorded the images, or by scanning photographs using a scanner (not shown). [0029]
  • The input data defining the positions and orientations at which the images were recorded may be generated in any of a number of different images. For example, the images may be processed as described in EP-A-0898245 or WO-A-01/39124 to calculate the relative positions and orientations based on matching features in the images, or the input images may be generated using a plurality of cameras surrounding the subject object with each camera having a known, fixed position and orientation which is input to the processing apparatus [0030] 2. Alternatively, a fixed camera may be used to image the subject object mounted on a turntable having position encoders which output the precise rotation of the turntable. Knowing the fixed position of the camera and the turntable angle, the relative position and orientation of each image of the subject object can be calculated.
  • The input data defining the intrinsic camera parameters may be input by a user using a user input device [0031] 6.
  • Image data segmenter [0032] 50 is arranged to process each input image to segment image data corresponding to the subject object from other image data in the image.
  • Surface modeller [0033] 60 is arranged to process the segmented image data produced by image data segmenter 50 and the input data defining the positions and orientations at which the images were recorded and the imaging parameters, to generate data defining a 3D computer model representing the actual surfaces of the subject object in the input images.
  • Surface modeller [0034] 80 comprises a wireframe mesh generator 70 and a mesh refiner 80.
  • Wireframe mesh generator [0035] 70 is arranged to process the input data to back-project the silhouette of the subject object in each input image to form a respective polyhedron in 3D space for each silhouette, and to generate a polygon mesh defining the volume of 3D space constrained by the polyhedra—that is, the visual hull of the subject object.
  • Mesh refiner [0036] 80 is arranged to process the polygon mesh generated by the wireframe mesh generator 70 to remove convex and concave artefacts, thereby improving the accuracy of the 3D computer model.
  • Display processor [0037] 90, under the control of central controller 20, is arranged to display images and instructions to a user via display device 4 during processing to generate the 3D computer model. In addition, under the control of central controller 20, display processor 90 is also arranged to display images of the 3D computer model of the object from a user-selected viewpoint by rendering the surface model data generated by surface modeller 60.
  • Output data store [0038] 100 is arranged to store the data defining the surface model generated by surface modeller 60. Central controller 20 is arranged to control the output of data from output data store 100, for example as data on a storage device, such as disk 112, and/or as a signal 114. A recording of the surface model generated by surface modeller 60 may be made by recording the output signal 114 either directly or indirectly using recording apparatus (not shown).
  • FIG. 2 shows the processing operations performed by processing apparatus [0039] 2 to process input data in this embodiment.
  • Referring to FIG. 2, at step S[0040] 2-2, central controller 20 causes display processor 90 to display a message on display device 4 requesting the user to input data for processing.
  • At step S[0041] 2-4, data input by the user in response to the request at step S2-2 is stored in the input data store 40. More particularly, in this embodiment, the input data comprises image data defining images of at least one subject object recorded at different relative positions and orientations, data defining the relative positions and orientations at which the input images were recorded, and data defining the intrinsic parameters of the camera which recorded the input images, that is the aspect ratio, focal length, principal point (the point at which the optical axis intersects the imaging plane), the first order radial distortion coefficient, and the skew angle (the angle between the axes of the pixel grid).
  • Thus, referring to FIG. 3, at this stage in the processing, processing apparatus [0042] 2 has data stored therein defining a plurality of images 200-270 of a subject object 300, data defining the relative positions and orientations of the images 200-270 in 3D space, and data defining the imaging parameters of the images 200-270, which defines, inter alia, the focal point positions 310-380 of the images.
  • Referring again to FIG. 2, at step S[0043] 2-6, image data segmenter 50 processes each input image stored at step S2-4 to segment image data representing the subject object form other image data (background image data). This processing is performed using a conventional image segmentation method, for example as described in part 2.2 of Annex A of GB-A-2358307.
  • Referring to FIG. 4, at this stage in the processing, therefore, the silhouette [0044] 400-470 of the subject object 300 in each input image has been calculated, and the relative position and orientation of each silhouette in 3D space is known from the input data (because each silhouette 400-470 lies in the two-dimensional plane of an input image). In addition, the position of the focal point 310-380 for each input image is also known from the input data.
  • At step S[0045] 2-8, wireframe mesh generator 70 processes the data generated by image data segmenter 50 defining the silhouettes 400-470 of the subject object 300 together with the input data stored at step S2-4 to generate data defining a mesh of connected polygons representing the visual hull of the subject object 300.
  • More particularly, referring to FIG. 4, each silhouette [0046] 400-470 defines, together with the focal point position 310-380 of the camera when the image in which the silhouette is situated was recorded, an infinite cone in three-dimensional space which touches the surface of the subject object 300 at (as yet unknown) points in the three-dimensional space (because the silhouette defines the outline of the subject object surface in the image). This is illustrated in FIG. 4 for silhouettes 410, 450 and 470, which, together with the focal point positions 320, 360 and 380 define respective infinite cones 500, 510 and 520 each having the same cross-sectional shape as the corresponding silhouette.
  • Wireframe mesh generator [0047] 70 performs processing at step S2-8 to determine the volume of 3D space defined by the intersection of the infinite cones defined by all of the silhouettes 400-470, and to represent the intersection volume by a mesh of connected planar polygons.
  • In this embodiment, this processing may be carried out using the technique described in the proprietor's co-pending US patent application 10/164435 (the full contents of which are incorporated herein by cross-reference), or may be carried out using a conventional method, for example such as that described in “A Volumetric Intersection Algorithm for 3D-Reconstruction Using a Boundary-Representation” by Martin Löhlein at http://i31www.ira.uka.de/diplomarbeiten/da_martin_loehlein/Reconstruction.html or as described in “An Algorithm for Determining the Intersection of Two Simple Polyhedra” by M. Szilvasi-Nagy in Computer Graphics Forum 3 (1984) pages 219-225. [0048]
  • Alternatively, wireframe mesh generator [0049] 70 may perform shape-from-silhouette processing for example as described in “Looking to build a model world: automatic construction of static object models using computer vision” by Illingsworth and Hilton in Electronics and Communication Engineering Journal, June 1998, pages 103-113, or “Automatic reconstruction of 3D objects using a mobile camera” by Niem in Image and Vision Computing 17 (1999) pages 125-134. In these methods the intersections of the silhouette cones are calculated and used to generate a “volume representation” of the subject object made up of a plurality of voxels (cuboids). More particularly, 3D space is divided into voxels, and the voxels are tested to determine which ones lie inside the volume defined by the intersection of the silhouette cones. Voxels inside the intersection volume are retained to define a volume of voxels representing the object. The volume representation is then converted into a surface model comprising a mesh of connected polygons.
  • Referring to FIG. 5[0050] a, the result of the processing performed by wireframe mesh generator 70 is data defining a 3D computer model comprising a polygon mesh 500 representing the visual hull of the subject object 300. In this embodiment, each polygon in the mesh 500 is a triangle, and comprises three edges and three vertices in a conventional manner.
  • The inventor in the present case has realised that one problem which often occurs with a visual hull polygon mesh is that convexities and concavities appear in the computer model which do not exist on the subject object [0051] 300 itself. More particularly, the inventor has identified two types of convex and concave artefacts which are a particular problem because they decrease the accuracy of the polygon mesh representing subject object 300. These are convex artefacts on top of planar surfaces forming a “dome” on the planar surface, and convex and concave artefacts in high curvature regions forming “creases” and “folds” in the surface that are not present on the subject object 300. An example of the first type of artefact is illustrated in FIGS. 5a and 5 b, which show a convex projection 510 which appears in the polygon mesh on a surface which is actually flat in the subject object 300 (the flat top surface of a cup). These artefacts occur because the polygon mesh is generated from only a limited number of silhouettes 410-470 which intersect to constrain the volume of 3D space occupied by the subject object 300, and because the 3D computer model represents the largest object in volume that could generate the input image data.
  • Accordingly, referring again to FIG. 2, at step S[0052] 2-10, mesh refiner 80 performs processing to refine the polygon mesh 500 generated by wireframe mesh generator 70, to remove the concave and convex artefacts in the mesh.
  • FIG. 6 shows the processing operations performed by mesh refiner [0053] 80 at step S2-10.
  • Referring to FIG. 6, at step S[0054] 6-2, mesh refiner 80 projects the polygon mesh 500 generated by wireframe mesh generator 70 into each respective input image 200-270.
  • In addition, in each input image [0055] 200-270, mesh refiner 80 sets the value of each pixel to either 0 or 1 to indicate whether or not the pixel is part of the projected polygon mesh (the value 1 indicating in this embodiment that the pixel is part of the projected mesh). In this way, a binary reference silhouette image is generated for each input image 200-270.
  • It should be noted that the pixel values defining the scene in the input images [0056] 200-270 stored at step S2-4 are of no relevance to the processing at step S6-2 since the purpose of this processing is to generate a binary reference silhouette image. Accordingly, the input pixel values stored at step S2-4 can be discarded before step S6-2 is performed (by setting all of the pixels to 0 or 1 in each input image).
  • At steps S[0057] 6-4 to S6-18, mesh refiner 80 performs processing to test edges of polygons in the polygon mesh 500 to determine whether the edges can be removed from the polygon mesh without compromising the consistency of the 3D computer model with the reference silhouettes generated at step S6-2. More particularly, as will be explained below, mesh refiner 80 tests whether a polygon edge can be removed by temporarily removing the edge to give an amended polygon mesh, projecting the amended polygon mesh into each input image to give projected silhouettes, and testing the resulting projected silhouettes against the reference silhouettes generated at step S6-2. If, for any image, the difference between a projected silhouette and the corresponding reference silhouette is greater than a predetermined threshold, then the polygon edge is retained in the polygon mesh (because the polygon mesh without it would be inconsistent with the silhouette data), otherwise the edge is permanently removed from the polygon mesh.
  • In this way, edges which have no effect on the accuracy of the 3D computer model are removed, resulting in a “simplified” polygon model (that is, one which contains fewer edges than the original model) that is consistent with the silhouette data. Consequently, this results in convex and concave artefacts being removed because, when the polygon edges defining these artefacts are removed, it is found in the subsequent tests that the resulting polygon mesh is still consistent with the reference silhouettes. [0058]
  • The processing operations performed by mesh refiner [0059] 80 at steps S6-4 to S6-16 will now be described in detail.
  • At step S[0060] 6-4, mesh refiner 80 calculates a collapse cost score for each edge in the polygon mesh 500. In this embodiment, this is carried out in accordance with a conventional edge decimation algorithm, for example as described in the article “A Simple Fast and Effective Polygon Reduction Algorithm” published at pages 44-49 of the November 1998 issue of Game Developer Magazine (publisher CMP Media, Inc) or as described in “Progressive Meshes” by Hoppe, Proceedings SIGGRAPH 96, pages 99-108.
  • More particularly, in accordance with the decimation algorithm, a collapse cost score is calculated for each edge in the polygon mesh [0061] 500 defining a measure of the effect that the edge's removal will have on the overall polygon mesh—the higher the score, the greater the effect the removal of the edge will have on the polygon mesh 500. This collapse cost score is calculated for each edge at step S6-4 in a conventional way, based on the edge's length and the flatness of the surface around the edge, and is independent of whether the polygon mesh without the edge is consistent with the reference silhouettes generated at step S6-2 (this consistency being tested in subsequent processing in this embodiment).
  • At step S[0062] 6-6, mesh refiner 80 selects the next “best” edge UV in the polygon mesh as a candidate edge to collapse (this being the first “best” edge the first time step S6-6 is performed). More particularly, mesh refiner 80 selects the edge having the lowest calculated collapse cost score as a candidate edge to collapse (since the removal of this edge should have the least effect on the polygon mesh).
  • At step S[0063] 6-8, mesh refiner 80 determines whether the collapse cost score associated with the candidate edge selected at step S6-6 is greater than a predetermined threshold value. On the first iteration of step S6-8, the collapse cost score associated with the candidate edge will be less than the predetermined threshold value. However, as will be explained below, when it is determined in subsequent processing that an edge should not be removed from the polygon mesh (because the resulting polygon mesh is inconsistent with one or more of the reference silhouettes generated at step S6-2) mesh refiner 80 sets the collapse cost score associated with the edge to a large predetermined value. The threshold value used in the test at step S6-8 is set to be below the collapse cost score set for an edge which is inconsistent with the reference silhouettes and therefore should not be removed from the polygon mesh. Accordingly, when it is determined at step S6-8 on subsequent iterations that the collapse cost score associated with the candidate edge is greater than the predetermined threshold, the processing has reached a stage where all edges which can be removed from the polygon mesh leaving the polygon mesh consistent with the reference silhouettes have already been removed, and no further edges can be removed. This is because the edge selected at step S6-6 as the candidate edge is the edge with the lowest collapse cost score, and accordingly if the collapse cost score is determined to be greater than the predetermined threshold at step S6-8, then the collapse cost score associated with all remaining edges will be greater than the predetermined threshold. In this case, the refinement of the 3D computer model is complete, and processing returns to step S2-12 in FIG. 2.
  • On the other hand, when it is determined at step S[0064] 6-8 that the collapse cost score associated with the candidate edge is not greater than the predetermined threshold, processing proceeds to step S6-10, at which mesh refiner 80 determines whether the refined polygon mesh resulting from collapsing the candidate edge is still consistent with the reference silhouettes in the input images (these being the reference silhouettes generated at step S6-2).
  • FIG. 7 shows the processing operations performed by mesh refiner [0065] 80 at step S6-10.
  • Referring to FIG. 7, at step S[0066] 7-2, mesh refiner 80 refines the polygon mesh. In this embodiment, this comprises setting the position of vertex “U” on the candidate edge selected at step S6-6 to the position of vertex “V” (with the effect that the candidate edge is collapsed and therefore removed from the polygon mesh).
  • At steps S[0067] 7-4 to S7-18, mesh refiner 80 tests the refined polygon mesh generated at step S7-2 against each reference silhouette previously generated at step S6-2, to determine whether the refined polygon mesh is consistent with all the reference silhouettes.
  • More particularly, at step S[0068] 7-4, mesh refiner 80 considers the next input image (this being the first input image the first time step S7-4 is performed).
  • At step S[0069] 7-6, mesh refiner 80 projects the refined polygon mesh generated at step S7-2 into the input image selected at step S7-4, and sets the value of each pixel in the input image to 0 or 1 to define whether or not the pixel is part of the projected silhouette (the value 1 indicating that the pixel is part of the silhouette in this embodiment). In this way, a refined silhouette image is generated for the input image defining the silhouette of the refined polygon mesh generated at step S7-2 as seen from the input image (that is, as seen by a camera having the position and orientation of the input image and the imagining parameters defined in the input data stored at step S2-4). As in step S6-2, the pixel values defining the scene in the input images 200-270 stored at step S2-4 are of no relevance to the processing at step S7-6, and can therefore be discarded before step S7-6 is performed (by setting all of the pixels to 0 or 1 in each input image).
  • At step S[0070] 7-8, mesh refiner 80 compares the refined silhouette image generated at step S7-6 with the reference silhouette image previously generated at step S6-2 for the same input image. More particularly, in this embodiment, mesh refiner 80 compares the refined silhouette image and the reference silhouette image on a pixel-by-pixel basis to generate a difference image. When carrying out a comparison of a pixel from the refined silhouette image with a pixel from the reference silhouette image, mesh refiner 80 sets the value of the corresponding pixel in the difference image to 0 if the pixel in the refined silhouette image and the corresponding pixel in the reference silhouette image have the same value—that is, if both pixels have the value 0 indicating that they are not part of the refined silhouette or the reference silhouette, or if both pixels have the value 1 indicating that they are part of the refined silhouette and reference silhouette. On the other hand, mesh refiner 80 sets the value of the pixel in the difference image to 1 if the value of the pixel in the reference silhouette image has a different value to the corresponding pixel in the refined silhouette image. In this way, pixels in the difference image are set to the value 1 at every position where the refined silhouette and reference silhouette do not overlap. Consequently, pixel values of 1 in the difference image define positions where the refined silhouette is inconsistent with the reference silhouette.
  • As will now be described, in this embodiment, based on the number of pixels in the difference image having the value 1, mesh refiner [0071] 80 determines whether the refined silhouette is inconsistent with the reference silhouette (with the consequence that the edge collapse made at step S7-2 should not be made in the final 3D computer model if the silhouettes are inconsistent).
  • More particularly, at step S[0072] 7-10, mesh refiner 80 erodes the pixels having the value 1 in the difference image generated at step S7-8 by a predetermined number of pixels. In this embodiment, mesh refiner 80 erodes the difference image by three pixels, and the erosion processing is carried out in a conventional manner, for example as described in Digital Image Processing, Second Edition, by Pratt, ISBN 0-471-85766-1, page 457.
  • At step S[0073] 7-12, mesh refiner 80 determines whether there are any pixels of value 1 remaining in the difference image after the erosion processing at step S7-10 has been carried out.
  • If it is determined at step S[0074] 7-12 that there is at least one pixel of value 1 remaining in the eroded difference image, then at step S7-14, mesh refiner 80 returns an “inconsistent” result, indicating that the refinement to the polygon mesh by collapsing the candidate edge at step S7-2 has resulted in the polygon mesh becoming inconsistent with a reference silhouette.
  • On the other hand, if it is determined at step S[0075] 7-12 that there are no pixels of value 1 remaining in the eroded difference image, then mesh refiner 80 determines that the refined polygon mesh generated at step S7-2 is consistent with the reference silhouette for the input image currently being considered, and processing proceeds to step S7-16 to consider further input images.
  • It will thus be understood that, by performing the erosion processing at step S[0076] 7-10 and counting at step S7-12 the number of pixels of value 1 in the eroded image, instead of counting the number of pixels of value 1 in the difference image itself generated at step S7-8, small, inconsequential differences between the refined silhouette and the reference silhouette are disregarded, increasing the accuracy and reliability of the refinement processing.
  • At step S[0077] 7-16, mesh refiner 80 determines whether there is another input image to be processed. Steps S7-4 to S7-16 are repeated until each input image has been processed in the way described above.
  • In this way, the refined polygon mesh generated at S[0078] 7-2 is tested against the reference silhouette for each input image.
  • When it is determined at step S[0079] 7-16 that the refined polygon mesh has been tested against all of the input images, processing proceeds to step S7-18, at which mesh refiner 80 returns a “consistent” result indicating that the refinement made to the polygon mesh at step S7-2 by collapsing the candidate edge is consistent with all of the reference silhouettes.
  • Following the processing at step S[0080] 7-18, or S7-14, mesh refiner 80 performs processing at step S7-20 to return the polygon mesh to the configuration it had before step S7-2 was performed. Accordingly, in this embodiment, mesh refiner 80 returns the vertex U (which was temporarily moved at step S7-2) to its original position.
  • Referring again to FIG. 6, if an “inconsistent” result is returned at step S[0081] 7-14 when step S6-10 is performed, then processing proceeds to step S6-12, at which mesh refiner 80 sets the collapse cost score associated with the candidate edge selected at step S6-6 to a predetermined value which is larger than the threshold value used in the test at step S6-8. In this way, if the candidate edge is subsequently selected again at step S6-6, processing will not proceed from step S6-8 to step S6-10 to test again whether the edge can be removed from the polygon mesh. Instead, the refinement processing will end because it will be determined at step S6-8 that the collapse cost score is greater than the predetermined threshold.
  • Following step S[0082] 6-12, at step S6-14, mesh refiner 80 sets an “invalid” flag for the candidate edge. The purpose of this flag will be explained below.
  • Referring again to step S[0083] 6-10, if a “consistent” result is returned at step S7-18, then processing proceeds to step S6-16, at which the candidate edge selected at step S6-6 is collapsed within the polygon mesh. In this embodiment, the edge collapse is carried out in a conventional way, for example as described in the article “A Simple Fast and Effective Polygon Reduction Algorithm” published at pages 44-49 of the November 1998 issue of Game Developer Magazine (publisher CMP Media, Inc) or as described in “Progressive Meshes” by Hoppe, Proceedings SIGGRAPH 96, pages 99-108, and results in the removal of two triangular polygons, one edge and one vertex from the polygon mesh. The effect of the processing at step S6-10 on the shape of the polygon mesh is the same as the effect of the processing at step S7-2, which was subsequently tested for consistency with the reference silhouettes at steps S7-18. However, the processing performed at step S6-10 includes “housekeeping” functions to reduce the number of triangles etc in the polygon mesh as a result of the edge collapse, thereby reducing the amount of data defining the polygon mesh.
  • Following step S[0084] 6-16, at step S6-18, mesh refiner 80 reads the edge flags associated with the edges remaining in the polygon mesh to determine if any have been set to “invalid” at step S6-14, and performs processing to update the collapse cost scores for the edges remaining in the polygon mesh.
  • More particularly, in this embodiment, mesh refiner [0085] 80 updates the cost scores in accordance with the conventional decimation algorithm used at step S6-4, but ensures that the collapse cost score for each edge having an “invalid” flag (set at step S6-14) remains above the predetermined threshold used in the test at step S6-8.
  • Steps S[0086] 6-6 to S6-18 are repeated to select edges in the polygon mesh and test them to determine whether they can be removed, until it is determined at step S6-8 that every edge remaining in the polygon mesh has a collapse cost score greater than the predetermined threshold. When this situation is reached, the removal of any edge in the polygon mesh would cause the mesh to be inconsistent with at least one reference silhouette generated at step S6-2, and accordingly the refinement processing ends.
  • Referring again to FIG. 2, at step S[0087] 2-12, central controller 20 outputs the data defining the 3D computer model of the subject object 300 generated by surface modeller 60 from output data store 100, for example as data stored on a storage device such as disk 112 or as a signal 114. In addition, or instead, central controller 20 causes display processor 90 to display on display device 4 an image of the 3D computer model in accordance with a viewpoint input by a user, for example using a user input device 6.
  • Second Embodiment [0088]
  • In the first embodiment described above, the processing performed by mesh refiner [0089] 80 at step S2-10 to refine the polygon mesh to remove concave and convex artefacts comprises decimation processing to remove edges in the polygon mesh without affecting the resulting polygon mesh's consistency with the silhouettes of the subject object.
  • However, mesh refiner [0090] 80 may be arranged to perform other types of processing at step S2-10 to refine the polygon mesh to remove concave and convex artefacts.
  • For example, a second embodiment will now be described in which mesh refiner [0091] 80 performs processing to smooth the polygon mesh without affecting the resulting mesh's consistency with the silhouettes of the subject object.
  • The components of the second embodiment and the processing operations performed thereby are the same as those in the first embodiment, with the exception of the processing performed by mesh refiner [0092] 80 at step S2-10 in FIG. 2. Accordingly, only the processing performed at step S2-10 will be described below.
  • FIG. 8 shows the processing operations performed by mesh refiner [0093] 80 at step S2-10 in the second embodiment to refine the polygon mesh generated by wire frame mesh generator 70 at step S2-8 to remove concave and convex artefacts therein.
  • Referring to FIG. 8, at step S[0094] 8-2 (which corresponds to step S6-2 in FIG. 6), mesh refiner 80 projects the polygon mesh 500 generated by wire frame mesh generator 70 into each respective input image 200-270.
  • In addition, in each input image [0095] 200-270, mesh refiner 80 sets the value of each pixel to either 0 or 1 to indicate whether or not the pixel is part of the projected polygon mesh (the value 1 indicating in this embodiment that the pixel is part of the projected mesh). In this way, a binary reference silhouette image is generated for each input image 200-270.
  • At step S[0096] 8-4, mesh refiner 80 sets the value of a counter to 0.
  • At steps S[0097] 8-6 to S8-18, mesh refiner 80 performs processing to test vertices of polygons in the polygon mesh 500 to determine whether the vertices can be moved to smooth the polygon mesh without compromising the consistency of the 3D computer model with the reference silhouettes generated at step S8-2. More particularly, as will be explained below, mesh refiner 80 tests whether a polygon vertex can be moved by temporarily moving the vertex to give an amended polygon mesh, projecting the polygon mesh into each input image to give projected silhouettes, and testing the resulting projected silhouettes against the reference silhouettes generated at step S8-2. If the difference between a projected silhouette and the corresponding reference silhouette is greater than a predetermined threshold, then the polygon vertex is not moved (because the polygon mesh would then be inconsistent with the silhouette data), otherwise the vertex is moved.
  • In this way, polygon vertices are moved to give a “smoothed” polygon model that is consistent with the silhouette data. Consequently, this results in convex and concave artefacts being removed because, when the polygon vertices defining these artefacts are moved, it is found in the subsequent tests that the resulting polygon mesh is still consistent with the silhouette data. However, small scale features that are apparent in the silhouettes are retained in the polygon mesh and not lost therefrom. [0098]
  • The processing operations performed by mesh refiner [0099] 80 at steps S8-6 to S8-18 will now be described in detail.
  • At step S[0100] 8-6, mesh refiner 80 selects at random a polygon vertex from the polygon mesh 500.
  • At step S[0101] 8-8, mesh refiner 80 calculates a displacement direction and distance for the vertex selected at step S8-6 to give a locally smoothed polygon mesh at the selected vertex. More particularly, in this embodiment, mesh refiner 80 calculates a displacement direction and displacement distance d for a selected xi vertex using the following equation: d _ = c m j ɛ N ( i ) x _ j - x _ i
    Figure US20030063086A1-20030403-M00001
  • where x=(x, y, z) [0102]
  • x[0103] j are the neighbours of the selected vertex xi (that is the vertices connected to xi by a single edge in the polygon mesh)
  • N(i) is the set of neighbour vertices of the selected vertex [0104]
  • m is the number of vertices in the set N(i) [0105]
  • c is a constant less than 1 (set to 0.6 in this embodiment) [0106]
  • This displacement is an approximation to the Laplacian operator at the selected vertex, and is referred to in the art as an “umbrella” operator. The umbrella operator is further described in “Implicit Fairing of Irregular Meshes using Diffusion and Curvature Flow” by Desbrun et al in SIGGRAPH99 ACM ISBN 0-201-48560 pages 317-324. [0107]
  • At step S[0108] 8-10, mesh refiner 80 determines whether the refined polygon mesh resulting from moving the vertex selected at step S8-6 in the direction and distance calculated at step S8-8 is still consistent with the reference silhouettes in the input images (these being the reference silhouettes generated at step S8-2).
  • The processing operations performed at step S[0109] 8-10 in the second embodiment by mesh refiner 80 are the same as those performed in the first embodiment at step S6-10, which are described above with reference to FIG. 7.
  • More particularly, referring again to FIG. 7, at step S[0110] 7-2, mesh refiner 80 refines the polygon mesh. In the second embodiment, this comprises moving the vertex selected at step S8-6 in the displacement direction calculated at step S8-8 by the displacement distance calculated at step S8-8.
  • At steps S[0111] 7-4 to S7-18, mesh refiner 80 tests the refined polygon mesh generated at step S7-2 against each reference silhouette previously generated at step S8-2, to determine whether the refined polygon mesh is consistent with all the reference silhouettes. These processing operations have been described previously in the description of the first embodiment, and accordingly will not be described again here.
  • At step S[0112] 7-20, mesh refiner 80 performs processing to return the polygon mesh to the configuration it had before step S7-2 was performed. Accordingly, in this embodiment, mesh refiner 80 returns the vertex selected at step S8-6 (which was temporarily moved at step S7-2) to its previous position.
  • Referring again to FIG. 8, if an “inconsistent” result is returned at step S[0113] 7-14 when step SB-10 is performed, then processing proceeds to step S8-12, at which mesh refiner 80 increments the value of the counter previously set to 0 at step S8-4.
  • At step S[0114] 8-14, mesh refiner 80 determines whether the value of the counter is now greater than a predetermined value, which, in this embodiment, is set to 1000.
  • If it is determined at step S[0115] 8-14 that the value of the counter is greater than 1000, then the refinement of the polygon mesh is complete, and the refinement processing ends. On the other hand, if the counter value is not yet greater than 1000, processing returns to step S8-6 to select another vertex from the polygon mesh at random.
  • Referring again to step S[0116] 8-10, if a “consistent” result is returned at step S7-18, then processing proceeds to step S8-16, at which the vertex selected at step S8-6 is moved in the displacement direction calculated at step S8-8 by the distance calculated at step S8-8. In this way, the polygon mesh is refined in a way such that the resulting polygon mesh is still consistent with the reference silhouettes.
  • Following step S[0117] 8-16, at step S8-18, mesh refiner 80 resets the value of the counter to 0, and processing returns to step S8-6 to select another vertex from the polygon mesh.
  • Steps S[0118] 8-6 to S8-18 are repeated to select vertices from the polygon mesh and test them to determine whether they can be moved, until it is determined at step S8-14 that the value of the counter is greater than 1000. The counter value represents the number of consecutive times that a vertex move has been discarded because the resulting polygon mesh is inconsistent with at least one reference silhouette image generated at step S8-2. Consequently, in this embodiment, refinement processing ceases when 1000 consecutive vertex moves have been determined to be inconsistent with the reference silhouettes (the discarded vertex moves being consecutive because the counter value is reset to 0 at step S8-18 whenever an allowable vertex move is found).
  • Modifications and Variations [0119]
  • Many modifications and variations can be made to the embodiments described above within the scope of claims. [0120]
  • For example, in the embodiments described above, the input image data comprises “still” images of the subject object [0121] 300. However, the input images may comprise frames of image data from a video camera.
  • In the embodiments described above, at step S[0122] 2-4, data input by a user defining the intrinsic parameters of the camera is stored. However, instead, default values may be assumed for some, or all, of the intrinsic camera parameters, or processing may be performed to calculate the intrinsic parameter values in a conventional manner, for example as described in “Euclidean Reconstruction From Uncalibrated Views” by Hartley in Applications of Invariance in Computer Vision, Mundy, Zisserman and Forsyth eds, pages 237-256, Azores 1993.
  • In the embodiments described above, data is input to processing apparatus [0123] 2 and stored at step S2-4 defining the positions and orientation of the input images. However, instead, processing apparatus 2 may calculate the positions and orientations of the input images, for example by carrying out processing as described in EP-A-0898245 or WO-A-01/39124.
  • In the embodiments described above, the reference silhouettes, against which refinements to the polygon mesh are tested, are created at step S[0124] 6-2 and step S8-2 by projecting the unrefined polygon mesh 500 into each of the input images 200-270 previously used to construct the unrefined polygon mesh 500. However, instead, the reference silhouettes may be generated by projecting the unrefined polygon mesh 500 into only a subset of the input images 200-270, with subsequent refinements to the polygon mesh being tested by projecting the refined mesh into the same subset of images. In addition or instead, the reference silhouettes may be generated by projecting the unrefined polygon mesh 500 into images which are not those stored at step S2-4 and used at steps S2-6 and S2-8 to generate the unrefined polygon mesh 500. More particularly, a plurality of “theoretical” images may be defined, having any position, orientation, size and number of pixels (and which of course do not need to have pixel values to define a scene within the image). Projecting the unrefined polygon mesh 500 into each “theoretical” image gives a set of reference silhouettes against which refinements to the polygon mesh can be tested by projecting the refined polygon mesh into the theoretical images and performing processing as described in the embodiments above. Accordingly, which images are used to generate the reference silhouettes is unimportant.
  • In the embodiments described above, processing is performed at step S[0125] 6-2 and step S8-2 to project the unrefined polygon mesh 500 generated by wireframe mesh generator 70 into each image to generate reference silhouettes. Each refinement of the polygon mesh by mesh refiner 80 is then tested to determine whether it is consistent with the reference silhouettes. However, the reference silhouettes need not be generated using the processing at step S6-2 and S8-2. Instead, for example, the silhouettes generated at step S2-6 by segmenting the input image data may be used as the reference silhouettes (although this has been found in practice to reduce the accuracy of the refinements to the polygon mesh).
  • In the embodiments described above, in order to test whether a refinement made by mesh refiner [0126] 80 to the polygon mesh is consistent with each reference silhouette, the entire refined polygon mesh is projected into each image (step S7-6) and the resulting refined silhouette in each image is tested against the entire reference silhouette for the image (step S7-8). However, instead, only the part of the refined polygon mesh containing the refinement made at step S7-2 can be projected into each image and the resulting partial silhouette tested against only part of the reference silhouette for the image. For example, a volume of 3D space containing all of the amendments to the polygon mesh resulting from the refinement at step S7-2, but not the whole of the polygon mesh, may be projected at step S7-6 into the selected input image to give a partial refined silhouette image and also into the corresponding reference silhouette image so that the 2D projection defines a partial reference silhouette image. At step S7-8, the partial refined silhouette and the partial reference silhouette images may then be compared to produce a difference image.
  • In the embodiments described above, processing is performed at step S[0127] 7-20 to return a temporarily refined polygon mesh to its previous configuration following tests to determine whether the refined polygon mesh is consistent with the reference silhouettes. Subsequently, at step S6-16 or step S8-16, the amendment to the polygon mesh is re-introduced if the tests indicated that the refined polygon mesh was consistent with the reference silhouettes. Alternatively, however, steps S7-20, S6-16 and S8-16 may be omitted and instead processing may be performed after it is determined at step S6-10 or step S8-10 that the refined polygon mesh is inconsistent with the reference silhouettes, to return the polygon mesh to its previous configuration (that is, the configuration before step S7-2 was performed).
  • In the embodiments described above, processing apparatus [0128] 2 includes image data segmenter 50 and wireframe mesh generator 70 which process the input data stored at step S2-4 to generate a polygon mesh representing the visual hull of the subject object 300 using a shape-from-silhouette technique. However, image data segmenter 50 and wireframe mesh generator 70 may be omitted from processing apparatus 2 and, instead, the data input to processing apparatus 2 and stored at step S2-4 may comprise data defining a polygon mesh representing the visual hull of an object generated using a shape-from-silhouette technique, and data defining the relative positions and orientations of the planes containing the silhouettes from which the visual hull model was generated. Mesh refiner 80 and processing apparatus 2 would then perform processing to refine the input mesh model in the way described above.
  • In the embodiments described above, processing is performed by a computer using processing routines defined by programming instructions. However, some, or all, of the processing could, of course, be performed using hardware. [0129]

Claims (32)

  1. 1. A method of processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the method comprising:
    generating a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    performing refinement processing a plurality of times to attempt to refine the polygon mesh, each time by:
    amending at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    projecting at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    testing the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    allowing the amendment if the difference between the projection silhouette of the refined mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise rejecting the amendment.
  2. 2. A method according to claim 1, further comprising projecting the polygon mesh prior to the refinement processing into each reference image to generate the respective reference silhouette for each reference image.
  3. 3. A method according to claim 2, wherein at least some of the reference images comprise images used to generate the polygon mesh.
  4. 4. A method according to claim 1, wherein the testing of the projection silhouette of the refined mesh against at least part of a reference silhouette in a reference image comprises:
    comparing the projection silhouette of the refined mesh against the at least part of the reference silhouette on a pixel-by-pixel basis to generate a difference image comprising pixels identifying positions where the projection silhouette and reference silhouette do not overlap;
    performing erosion processing of the difference image; and
    determining the number of pixels identifying positions where the projection silhouette and reference silhouette do not overlap remaining in the difference image after the erosion processing;
    and wherein the amendment is allowed if the number of pixels remaining in the difference image after the erosion processing does not exceed a predetermined number, otherwise the amendment is rejected.
  5. 5. A method according to claim 1, wherein the amendment to give a refined polygon mesh comprises collapsing at least one edge in the polygon mesh.
  6. 6. A method according to claim 1, wherein the amendment to give a refined polygon mesh comprises moving at least one vertex in the polygon mesh.
  7. 7. A method of processing data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to attempt to refine the polygon mesh, the method comprising:
    amending at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    projecting at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    testing the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween;
    allowing the change to the polygon mesh resulting from the amendment if the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise rejecting the change to the polygon mesh; and
    repeating the processing a plurality of times.
  8. 8. A method according to claim 7, further comprising projecting the polygon mesh prior to the refinement processing into each reference image to generate the respective reference silhouette for each reference image.
  9. 9. A method according to claim 8, wherein at least some of the reference images comprise images used to generate the initial polygon mesh.
  10. 10. A method according to claim 1 or claim 7, further comprising generating a signal conveying the refined three-dimensional computer model.
  11. 11. A method according to claim 10, further comprising making a recording of the signal either directly or indirectly.
  12. 12. Apparatus operable to process data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the apparatus comprising:
    a polygon mesh generator operable to generate data defining a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    a polygon mesh refiner operable to perform refinement processing, the polygon mesh refiner comprising:
    a polygon mesh editor operable to amend at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    a polygon mesh projector operable to project at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    a silhouette tester operable to test the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    an amendment selector arranged to allow the amendment if the difference between the projection silhouette of the refined mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise to reject the amendment.
  13. 13. Apparatus according to claim 12, further comprising a reference silhouette generator operable to project the polygon mesh prior to processing by the polygon mesh refiner into each reference image to generate the respective reference silhouette for each reference image.
  14. 14. Apparatus according to claim 13, wherein at least some of the reference images comprise images used to generate the polygon mesh.
  15. 15. Apparatus according to claim 12, wherein the silhouette tester comprises:
    a difference image generator operable to compare the projection silhouette of the refined mesh against the at least part of the reference silhouette on a pixel-by-pixel basis to generate a difference image comprising pixels identifying positions where the projection silhouette and reference silhouette do not overlap;
    an erosion processor operable to perform erosion processing of the difference image; and
    a pixel counter operable to determine the number of pixels identifying positions where the projection silhouette and reference silhouette do not overlap remaining in the difference image after the erosion processing;
    and wherein the amendment selector is arranged to allow the amendment if the number of pixels remaining in the difference image after the erosion processing does not exceed a predetermined number, otherwise to reject the amendment.
  16. 16. Apparatus according to claim 12, wherein the polygon mesh editor is operable to collapse at least one edge in the polygon mesh.
  17. 17. Apparatus according to claim 12, wherein the polygram mesh editor is operable to move at least one vertex in the polygon mesh.
  18. 18. Apparatus operable to process data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to attempt to refine the polygon mesh, the apparatus comprising:
    a polygon mesh editor operable to amend at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    a projection silhouette generator operable to project at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    a difference calculator operable to test the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    an amendment selector arranged to allow the change to the polygon mesh resulting from the amendment if the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise to reject the change to the polygon mesh.
  19. 19. Apparatus according to claim 18, further comprising a reference silhouette generator operable to project the polygon mesh into each reference image to generate the respective reference silhouette for each reference image.
  20. 20. Apparatus according to claim 19, wherein at least some of the reference images comprise images used to generate the initial polygon mesh.
  21. 21. Apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the apparatus comprising:
    means for generating data defining a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    means for performing refinement processing, comprising:
    means for amending at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    means for projecting at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    means for testing the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    amendment selection means arranged to allow the amendment if the difference between the projection silhouette of the refined mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise to reject the amendment.
  22. 22. Apparatus for processing data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to attempt to refine the polygon mesh, the apparatus comprising:
    means for amending at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    means for projecting at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    means for testing the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    amendment selection means arranged to allow the change to the polygon mesh resulting from the amendment if the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image does not exceed a threshold, otherwise to reject the change to the polygon mesh.
  23. 23. A method of processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the method comprising:
    generating a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    performing refinement processing a plurality of times, each time by:
    amending at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    projecting at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    testing the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    determining amendments for the polygon mesh in dependence upon the differences between the projection silhouette of the refined mesh and the reference silhouette in each reference image.
  24. 24. A method of processing data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to change the polygon mesh, the method comprising:
    amending at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    projecting at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    testing the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween;
    determining what change to make to the polygon mesh in dependence upon the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image and making the determined change; and
    repeating the processing a plurality of times.
  25. 25. Apparatus operable to process data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the apparatus comprising:
    a polygon mesh generator operable to generate data defining a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    a polygon mesh refiner operable to perform refinement processing, the polygon mesh refiner comprising:
    a polygon mesh editor operable to amend at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    a polygon mesh projector operable to project at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    a silhouette tester operable to test the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    an amendment calculator operable to determine what amendments to make to the polygon mesh in dependence upon the difference between the projection silhouette of the refined mesh and the reference silhouette in each reference image.
  26. 26. Apparatus operable to process data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to change the polygon mesh, the apparatus comprising:
    a polygon mesh editor operable to amend at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    a projection silhouette generator operable to project at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    a difference calculator operable to test the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    an amendment calculator operable to determine what amendments to make to the polygon mesh in dependence upon the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image, and to make the determined amendments to the polygon mesh.
  27. 27. Apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations, to generate data defining a three-dimensional computer model of the object, the apparatus comprising:
    means for generating data defining a polygon mesh representing the surface of the object, the polygon mesh defining a volume of three-dimensional space constrained by the projection of the respective silhouette of the object from each image into the three-dimensional space; and
    means for performing refinement processing, comprising:
    means for amending at least one vertex or edge in the polygon mesh to give a refined polygon mesh having an amendment;
    means for projecting at least part of the refined polygon mesh containing the amendment into a plurality of reference images to give a respective projection silhouette of the refined mesh in each reference image;
    means for testing the projection silhouette of the refined mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    means for determining what amendments to make to the polygon mesh in dependence upon the difference between the projection silhouette of the refined mesh and the reference silhouette in each reference image.
  28. 28. Apparatus for processing data defining (i) a polygon mesh representing the surface of an object, the polygon mesh defining a volume of three-dimensional space constrained by the projection into the three-dimensional space of a respective silhouette of the object from each of a plurality of images recorded at different positions and orientations and (ii) the positions and orientations at which the images were recorded, to change the polygon mesh, the apparatus comprising:
    means for amending at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    means for projecting at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    means for testing the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    amendment selection means for determining what amendments to make to the polygon mesh in dependence upon the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image, and for making the determined amendments to the polygon mesh.
  29. 29. A method of processing data defining a polygon mesh representing the surface of an object to change the polygon mesh, the method comprising:
    amending at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    projecting at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    testing the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween;
    determining what change to make to the polygon mesh in dependence upon the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image and making the determined change.
  30. 30. Apparatus operable to process data defining a polygon mesh representing the surface of an object to change the polygon mesh, the apparatus comprising:
    a polygon mesh editor operable to amend at least one vertex or edge in the polygon mesh to give an amendment changing the polygon mesh;
    a projection silhouette generator operable to project at least part of the changed polygon mesh containing the amendment into each of a plurality of reference images to give a respective projection silhouette of the changed mesh in each reference image;
    a difference calculator operable to test the projection silhouette of the changed mesh in each reference image against at least part of a respective reference silhouette in the reference image to determine the differences therebetween; and
    an amendment calculator operable to determine what amendments to make to the polygon mesh in dependence upon the difference between the projection silhouette of the changed mesh and the reference silhouette in each reference image, and to make the determined amendments to the polygon mesh.
  31. 31. A storage device storing instructions for causing a programmable processing apparatus to become operable to perform a method as set out in any of claims 1, 7, 23, 24 and 29.
  32. 32. A signal carrying instructions for causing a programmable processing apparatus to become operable to perform a method as set out in any of claims 1, 7, 23, 24 and 29.
US10255178 2001-09-28 2002-09-26 3D computer model processing apparatus Expired - Fee Related US7079680B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
GB0123385.7 2001-09-28
GB0123385A GB2381429B (en) 2001-09-28 2001-09-28 3D computer model processing apparatus

Publications (2)

Publication Number Publication Date
US20030063086A1 true true US20030063086A1 (en) 2003-04-03
US7079680B2 US7079680B2 (en) 2006-07-18

Family

ID=9922907

Family Applications (1)

Application Number Title Priority Date Filing Date
US10255178 Expired - Fee Related US7079680B2 (en) 2001-09-28 2002-09-26 3D computer model processing apparatus

Country Status (2)

Country Link
US (1) US7079680B2 (en)
GB (1) GB2381429B (en)

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040104916A1 (en) * 2002-10-29 2004-06-03 Canon Europa N.V. Apparatus and method for generating texture maps for use in 3D computer graphics
US20040155877A1 (en) * 2003-02-12 2004-08-12 Canon Europa N.V. Image processing apparatus
US20040196294A1 (en) * 2003-04-02 2004-10-07 Canon Europa N.V. Generating texture maps for use in 3D computer graphics
US20050052452A1 (en) * 2003-09-05 2005-03-10 Canon Europa N.V. 3D computer surface model generation
US20050151751A1 (en) * 2003-09-18 2005-07-14 Canon Europa N.V. Generation of texture maps for use in 3D computer graphics
US20050212811A1 (en) * 2004-03-24 2005-09-29 Fujitsu Limited Three-dimensional drawing model generation method, three-dimensional model drawing method, and program thereof
US20060209064A1 (en) * 2005-01-21 2006-09-21 Seiko Epson Corporation Image data generator and printer
US7545384B2 (en) 2000-10-27 2009-06-09 Canon Kabushiki Kaisha Image generation method and apparatus
US20100061637A1 (en) * 2008-09-05 2010-03-11 Daisuke Mochizuki Image processing method, image processing apparatus, program and image processing system
US8570343B2 (en) * 2010-04-20 2013-10-29 Dassault Systemes Automatic generation of 3D models from packaged goods product images
US20130293539A1 (en) * 2012-05-04 2013-11-07 Intermec Ip Corp. Volume dimensioning systems and methods
WO2014014562A1 (en) * 2012-07-20 2014-01-23 Google Inc. Determining three-dimensional (3d) object data models based on object movement
US20140160121A1 (en) * 2012-12-07 2014-06-12 Donya Labs Ab Method for forming an optimized polygon based shell mesh
US9292969B2 (en) 2012-05-07 2016-03-22 Intermec Ip Corp. Dimensioning system calibration systems and methods
US20160232712A1 (en) * 2010-01-18 2016-08-11 Fittingbox Method and device for generating a simplified model of a real pair of spectacles
US9464885B2 (en) 2013-08-30 2016-10-11 Hand Held Products, Inc. System and method for package dimensioning
US9557166B2 (en) 2014-10-21 2017-01-31 Hand Held Products, Inc. Dimensioning system with multipath interference mitigation
US9595108B2 (en) * 2009-08-04 2017-03-14 Eyecue Vision Technologies Ltd. System and method for object extraction
US9636588B2 (en) 2009-08-04 2017-05-02 Eyecue Vision Technologies Ltd. System and method for object extraction for embedding a representation of a real world object into a computer graphic
US9752864B2 (en) 2014-10-21 2017-09-05 Hand Held Products, Inc. Handheld dimensioning system with feedback
US9762793B2 (en) 2014-10-21 2017-09-12 Hand Held Products, Inc. System and method for dimensioning
US9779276B2 (en) 2014-10-10 2017-10-03 Hand Held Products, Inc. Depth sensor based auto-focus system for an indicia scanner
US9784566B2 (en) 2013-03-13 2017-10-10 Intermec Ip Corp. Systems and methods for enhancing dimensioning
US9786101B2 (en) 2015-05-19 2017-10-10 Hand Held Products, Inc. Evaluating image values
US9823059B2 (en) 2014-08-06 2017-11-21 Hand Held Products, Inc. Dimensioning system with guided alignment
US9835486B2 (en) 2015-07-07 2017-12-05 Hand Held Products, Inc. Mobile dimensioner apparatus for use in commerce
US9841311B2 (en) 2012-10-16 2017-12-12 Hand Held Products, Inc. Dimensioning system
US9857167B2 (en) 2015-06-23 2018-01-02 Hand Held Products, Inc. Dual-projector three-dimensional scanner
US9897434B2 (en) 2014-10-21 2018-02-20 Hand Held Products, Inc. Handheld dimensioning system with measurement-conformance feedback
US9939259B2 (en) 2012-10-04 2018-04-10 Hand Held Products, Inc. Measuring object dimensions using mobile computer
US9940721B2 (en) 2016-06-10 2018-04-10 Hand Held Products, Inc. Scene change detection in a dimensioner
US10007858B2 (en) 2012-05-15 2018-06-26 Honeywell International Inc. Terminals and methods for dimensioning objects
US10025314B2 (en) 2016-01-27 2018-07-17 Hand Held Products, Inc. Vehicle positioning and object avoidance
US10060729B2 (en) 2014-10-21 2018-08-28 Hand Held Products, Inc. Handheld dimensioner with data-quality indication
US10066982B2 (en) 2015-06-16 2018-09-04 Hand Held Products, Inc. Calibrating a volume dimensioner
US10094650B2 (en) 2015-07-16 2018-10-09 Hand Held Products, Inc. Dimensioning and imaging items

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7319784B2 (en) * 2003-02-05 2008-01-15 National Research Council Of Canada Magnetic resonance spectroscopy using a conformal voxel
US7565005B2 (en) * 2003-02-18 2009-07-21 Canon Kabushiki Kaisha Three-dimensional data processing method, three-dimensional data processing program, and three-dimensional data processing system
US7142726B2 (en) * 2003-03-19 2006-11-28 Mitsubishi Electric Research Labs, Inc. Three-dimensional scene reconstruction from labeled two-dimensional images
JP4262013B2 (en) * 2003-07-31 2009-05-13 キヤノン株式会社 The image processing method and image generation device
JP4392507B2 (en) * 2006-11-08 2010-01-06 国立大学法人東京工業大学 3-dimensional surface generation method
US20100316270A1 (en) * 2007-12-20 2010-12-16 Koninklijke Philips Electronics N.V. 3d reconstruction of a body and of a body contour
US8081183B2 (en) * 2007-12-21 2011-12-20 Industrial Technology Research Institute Method for reconstructing three dimensional model
US8131063B2 (en) * 2008-07-16 2012-03-06 Seiko Epson Corporation Model-based object image processing
US8260038B2 (en) * 2009-02-25 2012-09-04 Seiko Epson Corporation Subdivision weighting for robust object model fitting
US8260039B2 (en) * 2009-02-25 2012-09-04 Seiko Epson Corporation Object model fitting using manifold constraints
US8204301B2 (en) * 2009-02-25 2012-06-19 Seiko Epson Corporation Iterative data reweighting for balanced model learning
US8208717B2 (en) * 2009-02-25 2012-06-26 Seiko Epson Corporation Combining subcomponent models for object image modeling

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706419A (en) * 1995-02-24 1998-01-06 Canon Kabushiki Kaisha Image capturing and processing apparatus and image capturing and processing method
US5917937A (en) * 1997-04-15 1999-06-29 Microsoft Corporation Method for performing stereo matching to recover depths, colors and opacities of surface elements
US20010056308A1 (en) * 2000-03-28 2001-12-27 Michael Petrov Tools for 3D mesh and texture manipulation
US20020050988A1 (en) * 2000-03-28 2002-05-02 Michael Petrov System and method of three-dimensional image capture and modeling
US20020164066A1 (en) * 2000-11-22 2002-11-07 Yukinori Matsumoto Three-dimensional modeling apparatus, method, and medium, and three-dimensional shape data recording apparatus, method, and medium
US20030085891A1 (en) * 2001-11-05 2003-05-08 Alexander Lyons Three-dimensional computer modelling

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6106466A (en) * 1997-04-24 2000-08-22 University Of Washington Automated delineation of heart contours from images using reconstruction-based modeling
DE69823116D1 (en) 1997-08-05 2004-05-19 Canon Kk Image processing method and apparatus
CN1421020B (en) 1999-11-23 2010-04-28 佳能株式会 The image processing device
GB2358307B (en) 1999-11-25 2004-03-24 Canon Kk Image processing apparatus

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706419A (en) * 1995-02-24 1998-01-06 Canon Kabushiki Kaisha Image capturing and processing apparatus and image capturing and processing method
US5917937A (en) * 1997-04-15 1999-06-29 Microsoft Corporation Method for performing stereo matching to recover depths, colors and opacities of surface elements
US20010056308A1 (en) * 2000-03-28 2001-12-27 Michael Petrov Tools for 3D mesh and texture manipulation
US20020050988A1 (en) * 2000-03-28 2002-05-02 Michael Petrov System and method of three-dimensional image capture and modeling
US20020164066A1 (en) * 2000-11-22 2002-11-07 Yukinori Matsumoto Three-dimensional modeling apparatus, method, and medium, and three-dimensional shape data recording apparatus, method, and medium
US20030085891A1 (en) * 2001-11-05 2003-05-08 Alexander Lyons Three-dimensional computer modelling

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7545384B2 (en) 2000-10-27 2009-06-09 Canon Kabushiki Kaisha Image generation method and apparatus
US7019754B2 (en) 2002-10-29 2006-03-28 Canon Europa N.V. Apparatus and method for generating texture maps for use in 3D computer graphics
US20040104916A1 (en) * 2002-10-29 2004-06-03 Canon Europa N.V. Apparatus and method for generating texture maps for use in 3D computer graphics
US20040155877A1 (en) * 2003-02-12 2004-08-12 Canon Europa N.V. Image processing apparatus
US20040196294A1 (en) * 2003-04-02 2004-10-07 Canon Europa N.V. Generating texture maps for use in 3D computer graphics
US7304647B2 (en) 2003-04-02 2007-12-04 Canon Europa N.V. Generating texture maps for use in 3D computer graphics
US20050052452A1 (en) * 2003-09-05 2005-03-10 Canon Europa N.V. 3D computer surface model generation
US20050151751A1 (en) * 2003-09-18 2005-07-14 Canon Europa N.V. Generation of texture maps for use in 3D computer graphics
US7528831B2 (en) 2003-09-18 2009-05-05 Canon Europa N.V. Generation of texture maps for use in 3D computer graphics
US7567249B2 (en) * 2004-03-24 2009-07-28 Fujitsu Limited Three-dimensional drawing model generation method, three-dimensional model drawing method, and program thereof
US20050212811A1 (en) * 2004-03-24 2005-09-29 Fujitsu Limited Three-dimensional drawing model generation method, three-dimensional model drawing method, and program thereof
US20060209064A1 (en) * 2005-01-21 2006-09-21 Seiko Epson Corporation Image data generator and printer
US20100061637A1 (en) * 2008-09-05 2010-03-11 Daisuke Mochizuki Image processing method, image processing apparatus, program and image processing system
US9636588B2 (en) 2009-08-04 2017-05-02 Eyecue Vision Technologies Ltd. System and method for object extraction for embedding a representation of a real world object into a computer graphic
US9595108B2 (en) * 2009-08-04 2017-03-14 Eyecue Vision Technologies Ltd. System and method for object extraction
US9569890B2 (en) * 2010-01-18 2017-02-14 Fittingbox Method and device for generating a simplified model of a real pair of spectacles
US20160232712A1 (en) * 2010-01-18 2016-08-11 Fittingbox Method and device for generating a simplified model of a real pair of spectacles
US8570343B2 (en) * 2010-04-20 2013-10-29 Dassault Systemes Automatic generation of 3D models from packaged goods product images
US20130293539A1 (en) * 2012-05-04 2013-11-07 Intermec Ip Corp. Volume dimensioning systems and methods
US9779546B2 (en) * 2012-05-04 2017-10-03 Intermec Ip Corp. Volume dimensioning systems and methods
US9292969B2 (en) 2012-05-07 2016-03-22 Intermec Ip Corp. Dimensioning system calibration systems and methods
US10007858B2 (en) 2012-05-15 2018-06-26 Honeywell International Inc. Terminals and methods for dimensioning objects
WO2014014562A1 (en) * 2012-07-20 2014-01-23 Google Inc. Determining three-dimensional (3d) object data models based on object movement
US8754887B2 (en) 2012-07-20 2014-06-17 Google Inc. Determining three-dimensional (3D) object data models based on object movement
US9939259B2 (en) 2012-10-04 2018-04-10 Hand Held Products, Inc. Measuring object dimensions using mobile computer
US9841311B2 (en) 2012-10-16 2017-12-12 Hand Held Products, Inc. Dimensioning system
US20140160121A1 (en) * 2012-12-07 2014-06-12 Donya Labs Ab Method for forming an optimized polygon based shell mesh
US9311749B2 (en) * 2012-12-07 2016-04-12 Donya Labs Ab Method for forming an optimized polygon based shell mesh
US9784566B2 (en) 2013-03-13 2017-10-10 Intermec Ip Corp. Systems and methods for enhancing dimensioning
US9464885B2 (en) 2013-08-30 2016-10-11 Hand Held Products, Inc. System and method for package dimensioning
US9823059B2 (en) 2014-08-06 2017-11-21 Hand Held Products, Inc. Dimensioning system with guided alignment
US9779276B2 (en) 2014-10-10 2017-10-03 Hand Held Products, Inc. Depth sensor based auto-focus system for an indicia scanner
US10060729B2 (en) 2014-10-21 2018-08-28 Hand Held Products, Inc. Handheld dimensioner with data-quality indication
US9752864B2 (en) 2014-10-21 2017-09-05 Hand Held Products, Inc. Handheld dimensioning system with feedback
US9762793B2 (en) 2014-10-21 2017-09-12 Hand Held Products, Inc. System and method for dimensioning
US9557166B2 (en) 2014-10-21 2017-01-31 Hand Held Products, Inc. Dimensioning system with multipath interference mitigation
US9897434B2 (en) 2014-10-21 2018-02-20 Hand Held Products, Inc. Handheld dimensioning system with measurement-conformance feedback
US9786101B2 (en) 2015-05-19 2017-10-10 Hand Held Products, Inc. Evaluating image values
US10066982B2 (en) 2015-06-16 2018-09-04 Hand Held Products, Inc. Calibrating a volume dimensioner
US9857167B2 (en) 2015-06-23 2018-01-02 Hand Held Products, Inc. Dual-projector three-dimensional scanner
US9835486B2 (en) 2015-07-07 2017-12-05 Hand Held Products, Inc. Mobile dimensioner apparatus for use in commerce
US10094650B2 (en) 2015-07-16 2018-10-09 Hand Held Products, Inc. Dimensioning and imaging items
US10025314B2 (en) 2016-01-27 2018-07-17 Hand Held Products, Inc. Vehicle positioning and object avoidance
US9940721B2 (en) 2016-06-10 2018-04-10 Hand Held Products, Inc. Scene change detection in a dimensioner

Also Published As

Publication number Publication date Type
GB0123385D0 (en) 2001-11-21 grant
GB2381429B (en) 2005-07-27 grant
US7079680B2 (en) 2006-07-18 grant
GB2381429A (en) 2003-04-30 application

Similar Documents

Publication Publication Date Title
Surazhsky et al. Explicit surface remeshing
Julius et al. D‐charts: Quasi‐developable mesh segmentation
Lazarus et al. Three-dimensional metamorphosis: a survey
Laine et al. Efficient sparse voxel octrees
US7538764B2 (en) System and method to obtain surface structures of multi-dimensional objects, and to represent those surface structures for animation, transmission and display
Duchaineau et al. ROAMing terrain: real-time optimally adapting meshes
US6069634A (en) System for rapidly deforming a graphical object
US6999073B1 (en) Method and system for generating fully-textured 3D
US6795069B2 (en) Free-form modeling of objects with variational implicit surfaces
US6518963B1 (en) Method and apparatus for generating patches from a 3D mesh model
US6023279A (en) Method and apparatus for rapidly rendering computer generated images of complex structures
US20040095357A1 (en) Image-based modeling and photo editing
US6307554B1 (en) Apparatus and method for generating progressive polygon data, and apparatus and method for generating three-dimensional real-time graphics using the same
Luebke et al. Level of detail for 3D graphics
Bittar et al. Automatic reconstruction of unstructured 3D data: Combining a medial axis and implicit surfaces
US20050140670A1 (en) Photogrammetric reconstruction of free-form objects with curvilinear structures
Jones et al. 3D distance fields: A survey of techniques and applications
US20030052875A1 (en) System and method to obtain surface structures of multi-dimensional objects, and to represent those surface structures for animation, transmission and display
US20080043021A1 (en) Three Dimensional Polygon Mesh Deformation Using Subspace Energy Projection
Jia et al. Inference of segmented color and texture description by tensor voting
US4888583A (en) Method and apparatus for rendering an image from data arranged in a constructive solid geometry format
Bradshaw et al. Sphere-tree construction using dynamic medial axis approximation
US5448686A (en) Multi-resolution graphic representation employing at least one simplified model for interactive visualization applications
US6285805B1 (en) System and method for finding the distance from a moving query point to the closest point on one or more convex or non-convex shapes
US20020085001A1 (en) Image processing apparatus

Legal Events

Date Code Title Description
AS Assignment

Owner name: CANON EUROPA N.V., NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BAUMBERG, ADAM MICHAEL;REEL/FRAME:013332/0608

Effective date: 20020823

CC Certificate of correction
FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

FEPP

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.)

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.)

FP Expired due to failure to pay maintenance fee

Effective date: 20180718