JP2010243478A - Method and apparatus for estimating 3d pose of 3d object in environment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a pose of a specular reflection object even when there is a change in illumination. <P>SOLUTION: In a processing step of the estimate of a 3D pose of a 3D specular reflection object 101 in an environment, a set of pairs of 2D reference images are generated using a 3D model of the object 101 and a set of poses of the object, wherein each pair of reference images is associated with one of the poses. Then, a pair of 2D input images of the object are acquired. A rough 3D pose of the object is estimated by comparing features in the pair of 2D input images and the features in each pair of 2D reference images using a rough cost function. The rough cost estimate is optionally refined using a fine cost function. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates generally to estimating the 3D pose of an object, and more particularly to estimating the 3D pose of a specular object.

Pose Estimation Three dimensional (3D) pose estimation determines the location and angular orientation of an object. A typical pose estimation method relies on several cues such as 2D texture images and 3D range images. Texture image based methods assume that the texture is invariant to environmental variations. However, this assumption is not true when there are illumination changes or shadows. Usually, most of these methods cannot handle specular objects.

  The range image based method utilizes 3D information that is independent of the appearance of the object and can overcome some of these problems. However, distance acquisition devices are more expensive than simple cameras.

Specular Object For some objects, it is very difficult to reconstruct the 3D shape. For example, it is difficult to recover the 3D shape of highly specular objects such as mirror objects or glossy metallic objects, and it is known that reliability is low.

  A cue by reflection is more sensitive to posture changes than a cue by texture or distance. Therefore, it becomes possible to estimate the posture parameter very accurately by the clue by reflection. However, it is not certain whether or not the clue due to reflection is applicable not only to refinement of posture but also to overall posture estimation, that is, object detection, object division, and rough object posture estimation.

  Prior art methods are usually based on appearance. Appearance is affected by lighting, shading, and scale. Therefore, it is difficult for these methods to overcome the related challenges such as partial occlusion, clutter scenes, and large posture variations. To address these issues, these methods use lighting-independent features such as points, lines, and silhouettes, or cost functions that are not affected by illuminance such as normalized cross-correlation (NCC) To do. However, for these methods to be successful, the object needs to be sufficiently textured. Severe illumination changes remain a problem, especially for specular objects.

  Various methods derive slight local shape information from identifying and tracking distorted reflections of the light source and known special features. Density measurements can also be obtained using the usual framework of optical path triangulation. However, these methods typically require accurate calibration and control of the environment surrounding the object, and in some cases, require many input images.

  Some methods for the reconstruction of specular objects do not require environmental calibration. These methods assume small environmental movements that cause specular flow on the image plane. These methods use specular flow to simplify speculation of specular reflection shapes in unknown complex illumination. However, a pair of linear partial differential equations must be solved, which usually requires an initial state that is not easily estimated in real-world applications.

  One method of estimating pose based on specular reflection uses a short image sequence and an initial pose estimate calculated by a standard template matching procedure. The diffuse component and the specular reflection component are separated for each frame, and an environment map is derived from the estimated specular reflection image. Next, the environment map and the image texture are simultaneously aligned to increase the accuracy of the pose estimation process.

  Embodiments of the present invention provide a method and system for estimating a 3D pose of a 3D specular object in an environment, implemented in a processor. The basis for the estimation is to match the features in the 2D image of the specular object acquired by the 2D camera. The image can be acquired by a conventional camera or a high dynamic range (HDR) camera. The HDR camera can accurately acquire a wide range of scene brightness within an image.

  In a pre-processing step, features are generated from a set of 3D models of objects and possible poses of objects. For each pose of the 3D model, the features are: (a) a single HDR image, (b) a reference image pair representing two different exposure settings, (c) a binary image acquired by the HDR image and a camera with a nonlinear intensity response. Can be a thresholded binary image, or (d) a specular flow image.

  Note that an image pair can be derived from a single HDR image. To increase accuracy, images can be acquired for a number of different exposures.

  Next, an input image pair of the object is acquired. This pair can be obtained by acquiring two images, one with short exposure and the other with long exposure, or a single HDR camera image through simulating short exposure and long exposure. Can be obtained from Input features are calculated from the input image. The initial 3D pose of the object is estimated by comparing the features in the input image pair with the features in the reference image pair using a first cost function.

  Optionally, a second cost function can be used to refine the 3D pose of the object. Note that if the database stores a large number of reference images for most possible poses, the initial pose estimate will be precise and accurate. For example, if the database contains about 1000 poses, the estimate is approximate, and if the database stores 1000000 poses, the estimate is more precise.

  In one embodiment, the feature is the specular intensity in the image. If three (RGB) channels are used, the specular intensity feature has a color. This makes the features more distinguishable. To build a 2D environment map, small mirror spheres are placed in the environment and image pairs are acquired, for example, one with short exposure and the other with long exposure, or, for example, with short exposure and Both are derived from a single HDR camera image through a long exposure simulation. A sphere can be placed in a scene with or without objects. Other features based on specular reflection are within the scope of the present invention.

  Each mirror sphere image is used to build a 2D environment map. A reference image pair is generated using this map. Thereafter, the reference image is compared with the input image pair, and the 3D posture of the specular object is estimated. Note that the environment map can be built during preprocessing or while the input image is acquired to adapt to changing illumination.

  In another embodiment, the feature is a specular flow in the image. Specular flow is a special case of optical flow. By causing motion in either the environment or the camera, a specular flow is generated for the set of 3D poses. An input specular flow is also calculated from the input image. Thereafter, the reference specular reflection flow image is compared with the input specular reflection flow image, and the 3D posture of the specular reflection object is estimated. As described above, a precise posture is estimated from a rough posture using an approximate cost function and a precise cost function.

  The present invention uses specular reflection to estimate the overall 3D pose of a 3D object using a 3D model of the object. The method can deal with difficult objects such as objects that have no texture and are highly specular. The method uses a simple matching cost function and optimization procedure, whereby the method can be implemented on a graphics processor unit (GPU) to improve performance.

4 is a flowchart of a method for estimating a 3D posture of a specular object using specular intensity information according to an embodiment of the present invention; 6 is a flowchart of a method for estimating a 3D posture of a specular object using specular flow information according to an embodiment of the present invention; FIG. 5 is a schematic diagram of stencil selection using incident light according to an embodiment of the present invention. It is an example of a reference image having pixels with high reliability and pixels with low reliability.

System and Method Overview FIG. 1 is a flow diagram of a method for estimating a 3D pose of an object 101 in an environment 102, as implemented in a processor 100. Here, the object has a specular reflection surface. In this embodiment, the feature is specular reflection intensity. When three (RGB) channels are used, the combined specular intensity has a color. A two-dimensional (2D) image of the environment is acquired by the camera 103. In one embodiment, the camera has a high dynamic range (HDR). Alternatively, the camera can use polarized light to estimate the specular component.

  The 3D pose is defined by the 3D translation vector (X, Y, Z) and the 3D Euler angles (μ, φ, σ) with respect to orientation in the camera coordinate system.

  In one application, a 3D pose is used to remove an object from the bin 106 using the robot arm 105. For example, in manufacturing applications, a container includes a plurality of identical objects that are picked up and manipulated one at a time according to their estimated posture. In this embodiment, it may be advantageous to mount the camera on a robot arm.

The object has a mirror-like surface like a glossy metallic object, has no texture, and is highly specularly reflective. Thus, the only data that can be used with this method is specular reflection on objects in 2D images. The distance between the object and the camera is Z ≒ Z _0. This distance can be used to determine a measure of projection. Furthermore, this distance can also be estimated using laser projection, a stereo camera, or any other conventional method.

  The method uses the low level features in the 2D image to estimate the 3D pose as follows. The method generates a reference image 125 from the 3D model 121 using a bi-directional reflection distribution function (BRDF) of expected surface reflectance for a number of possible posture deformations 122. Steps 110 and 120 can be a single preprocessing step. If the BRDF of the object is known, the image can be generated using the BRDF.

  The 2D input image 131 is acquired by the camera system and then compared with the reference image to determine the best matching 3D pose 151 of the specular object. This model can be a CAD / CAM model, a polygonal model, or any other suitable model.

Specular Intensity Features In one embodiment, a small mirror sphere is placed in the environment 102 with or without an object. Next, the acquired 2D image of the mirror sphere can be used to generate an environment map 111 representing illuminance information.

  In another embodiment, a set of environmental camera images is registered to create a mosaic (or full view) of the environment. The environment map 111 representing the environment illuminance information can be created using the mosaic or the entire view of the environment.

  This illuminance information can be used to generate a reference specular image 125 that is used for comparison with the input image (120).

  The ambient illuminance can include illumination of multiple light sources, such as ceiling lights, windows, or lights on the container. In addition, the lights can have various colors, such as red, green, and blue. In this case, the cost function is calculated independently for each light color. Next, an attitude is obtained so as to minimize the sum of the R cost function, the G cost function, and the B cost function.

  The ambient light can also be actively enhanced and manipulated using a light source, for example by projecting a pattern on the ceiling from one or more projectors.

Specular Flow Features In another embodiment, the method uses specular flow as a feature. Specular flow is defined as the optical flow caused by the movement of an object, camera, or environment. The specular flow does not depend on changing the illumination state, but on changing the movement, shape, and posture of the object. Therefore, the specular reflection flow can be used as a feature independent of illuminance for posture estimation.

Method Based on Environment Map As shown in FIG. 1, before estimating the 3D pose of an object, a pair of environment maps EL and ES 111 of the environment 102 are acquired by processing an image of a spherical mirror-like object (110 ). Each of these maps has a long exposure and a short exposure of, for example, about 1/4 second and 1/60 second. In the same long exposure and short exposure, input images IL and IS 131 are acquired (130). Alternatively, EL and ES images, and IL and IS images are obtained from a single image acquired using the HDR camera 103 followed by processing simulation via long exposure and short exposure. Can do.

Generation of Initial Posture Estimation Reference Specular Reflection Image A reference image 125 is generated for a number of predetermined postures 122 corresponding to possible postures of the object from the 3D model 121 of the object and a pair of environment maps EL and ES111.

For this purpose, the Euler angles are sampled uniformly and densely to define a large number of, for example, 25,000 poses. The reference images are RL and RS for various Euler angles (μ, φ, σ) at location (0, 0, Z ₀ ). By ignoring interreflection and self-occlusion, full specular images can be generated from EL and ES by applying reflection mapping. This is a special case of texture mapping.

  The reference image can also be generated by non-uniform sampling of posture. This can also depend on the attitude distribution of a given object.

  The reference specular image 125 depends on the 3D location and the orientation of the object relative to the camera. However, the camera has a small field of view 104 and the depth of the object is known. Therefore, the difference between the reference specular images generated from different 3D locations can be ignored. This is sufficient for the initial posture estimation 140. Note that the input images can also be obtained from multiple views for higher accuracy.

  The input image 131 is compared with the reference specular image 125 and the 3D posture 141 is estimated by solving the following (140).

However,

Represents an initial posture 141, C _R () is an approximate cost function 139 for comparison, an argument min is a function that returns an argument that generates a minimum value, and an inner minimum value is obtained before an outer minimum value. It is done. If the predetermined reference pose is sampled more precisely, eg, 1000000 poses, the initial pose estimate need not be considered approximate and need not be refined.

  The cost function 139 is as follows.

However, lambda is a control parameter, C 1 ₍₎ and C _{2 (),} respectively, the cost functions for the long-time exposure image and the short-time exposure image. To obtain these terms, the 3D translation vector (X, Y, Z ₀ ) is projected onto the 2D image plane and the reference image is moved to the projection point (x, y). Next, each translation reference image pair is compared to a corresponding input image pair.

  As used herein, the cost function measures the degree to which the input image features and reference image features (specular reflection intensity or specular reflection flow) are well matched to each reference orientation.

Cost Function Based on Highlight Pixel Usually, a specular image includes highlight pixels and non-highlight pixels. A highlight pixel corresponds to a light source having high intensity incident light, such as a lamp or window. Therefore, the pixel value is usually saturated.

Highlight pixels are used for the first term C ₁ (). Since an object has high specular reflectivity, highlight pixels can be extracted by applying a binarization process to a short-time exposure image (or HDR image) to create a binary image.

Using a binary image and the distance transform to construct the respective input highlight image and the reference highlight image corresponding distance image D _I and D _R. This distance transformation helps to match the binary images accurately. The cost function C ₁ () is defined as follows.

However, (u, v) is a pixel _{coordinate, N highlight} denotes the number of pixels summation is performed. The reference highlight pixels and their neighboring pixels are used as stencils for calculations.

Using a new threshold for creating a binary image, it should be understood that it is possible to generate two or more pairs of D _I picture and D _R image. The cost is changed to consider all such pairs simultaneously.

  The cost function based on highlights has the following advantages. First, because highlights are usually very slight in the input image, they can be used as a strong constraint on the location of the object. Second, the cost function is smoother than the cost function of the conventional cost function that uses all specular pixels. Third, because the highlight stencil contains only a very small number of pixels, this cost calculation can be done efficiently. Minimization can be performed using any suitable optimization procedure. In one embodiment, downhill simplex minimization is used. The downhill simplex minimization method converges well to the global minimum very quickly and stably.

Cost function based on all specular reflection pixels The second term C ₂ () considers all specular reflection pixels.

Here, NCC represents normalized cross correlation (NCC). Here, the object segmentation mask can be used as a stencil for the NCC. However, using only geometrically reliable specularly reflective pixels as stencils produces better results at run time.

  To handle a large dynamic range, prior to cost calculation, convert the pixel intensity to a new space, eg logarithmic response (or exponential response), thereby reducing (or increasing) the importance of specular pixels. It is understood that it can be done.

  As shown in FIG. 3, the geometric stencil selection is as follows. First, the incident ray for each pixel in the reference image

Is estimated. Where the reflected rays

And the surface normal of the reflected beam

Is known. From the law of reflection, the incident light is represented by:

Next, the reliability of the pixel information can be defined by considering the irradiation direction. As shown in FIG. 4 with respect to the reference image example 125, irradiation from _{i 1} has high reliability (401) irradiation from _{i 2} has a low reliability. Irradiation direction is the elevation angle of the camera coordinate system

And azimuth

Represented by

  Irradiation with a small elevation angle is usually less reliable than irradiation with a large elevation angle. This is due to changes in the environment map such as interreflection between specular objects and the use of different backgrounds in the environment. Finally, for the stencil of equation (4), only reliable specular pixels, that is, pixels with incident light at elevation angles greater than 90 degrees are used.

Overall procedure The overall method for posture estimation is as follows. First, a reference specular reflection image 125 is generated. For each possible pose 122, the optimal translation parameter is obtained. Any three end points of the input image are used as initial points for the downhill simplex method. The control parameter λ is changed from 0 to 1. This means that the translation is roughly optimized using only the highlight pixels and then improved by considering all specular pixels. After translation optimization, there are many translation optimization poses and their associated cost values. The minimum cost value is the optimum rotation parameter

Is for.

Optional Posture Refinement After estimating a rough initial posture 141 (140), the posture parameters can be further refined (150) by continuing to optimize the posture parameters. Since the translational posture has already been continuously optimized by the downhill simplex method in rough posture estimation, it is only necessary to refine only the rotational posture using the following cost function 149.

Here, R is a reference image obtained using the long exposure environment map EL. This optimization uses the steepest descent method.

Approach Based on Specular Flow FIG. 2 shows how optical flow is used as a feature for matching. Optical flow is usually an explicit pattern of movement in the environment caused by relative movement between the camera and the environment. In this embodiment, it is assumed that the optical flow is caused either by environmental movement or via camera movement.

  Two input images are generated by a defined small rotation of the environment around a known direction, eg, the viewing direction of the camera 103 (210). Alternatively, more than two images can be used.

  Next, the specular reflection flow between these two images is obtained to obtain an input specular reflection image I231 including a 2D displacement vector for each pixel. The specular flow is obtained using the block matching method.

  Usually, the specular flow can be caused by the movement of the object 101, the environment 102, or the camera 103. For simplicity of this description, only the movement of the environment is described, but it is assumed that the specular flow may be caused by the movement of the object 101, the environment 102, or the camera 103. Since the relative posture between the camera and the object is fixed, the specular reflection flow is observed only in the specular reflection pixel. Thus, this movement cue can be used to indicate whether there is a specular flow and to strongly constrain the location of the object.

Generation of Posture Estimation Reference Specular Reflection Flow For rough pose estimation 240, at each pose of the various poses 122, two or more specular reflection images for location (0, 0, Z ₀ ) are obtained as described above. Generate (220), but this time use a color-coded environment map that is slightly rotated, eg, ± 5 degrees. The color coded environment is simulated rather than measured. The color-coded environment makes it possible to determine an exact pixel correspondence between the two images, thereby easily and completely calculating the optical flow. A reference image R225 is generated using the resulting optical flow image.

Posture Optimization The reference image 225 is compared with the acquired (230) input specular reflection flow image I231, and the 3D posture 241 is estimated by minimizing the cost function 239 (240).

However, C ₁ () and C ₂ () are cost functions based on motion division and specular reflection flow, respectively. An optimization procedure is used to calculate translation and rotation estimates that minimize the cost function. In one embodiment, the translation (X, Y) is first optimized for each rotation using the downhill simplex method. The rotation is then optimized by comparing all cost values.

Cost Function Based on Motion Division Motion division is defined as a binary image that indicates whether there is a non-zero specular flow for each pixel. D _I and D _R represent distance conversion images constructed from motion division of the input image I 231 and the reference specular flow image R 225. The cost function C ₁ () is as follows.

However, the sum is executed with respect to the motion division pixels of the reference image R, and N _motion represents the number of such pixels. It should be understood that pose estimation and object segmentation can be performed simultaneously.

Cost function based on specular flow The second cost term C ₂ () in equation (6) is constructed by comparing the input image I (u, v) 231 with the translated reference image R225. Due to the noisy and textureless areas in actual applications, the input image contains many outliers. An outlier pixel is a pixel that does not match another (inlier) pixel in the image. Therefore, simple matching costs such as sum of squared differences (SSD) do not work well. Instead, the cost function is based on the number of pixels that are not outliers.

First, a pixel that is not an abnormal value is a pixel in which the difference between the input specular reflection flow vector I (u, v) and the reference specular reflection flow vector R is a small threshold, for example, less than 1.0. The cost function C ₂ () is as follows.

However, M is a set of pixels that are not abnormal values.

Overall Procedure The method based on the specular flow uses the same overall method as the method based on the specular intensity. The reference image 225 is generated using the model 121 and possible pose 122. An optimal translation is estimated for each reference image using the downhill simplex method. Here, the control parameter varies from 0 to 1. Next, all translation optimized postures are compared to determine the optimal rotation.

Optional Posture Refinement After estimating the approximate posture 241 (240), optionally refine the rotational posture parameters by minimizing the cost function 249 (250).

Here, R is a reference image having posture parameters (θ, φ, σ, X, Y), and N _mask represents the number of pixels in the stencil. A stencil is defined as an object segmentation mask.

  Although the invention has been described by way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the invention.

  Accordingly, it is the object of the appended claims to cover all modifications and variations that fall within the true spirit and scope of the invention.

Claims (42)

A method for estimating a 3D pose of a 3D object in an environment, the object having a specular surface and comprising a processor for performing the steps of the method, the method comprising:
Rendering a set of 2D reference image pairs using a 3D model of the object and a set of poses of the object, wherein each reference image pair is associated with one of the poses;
Obtaining a 2D input image pair of the object;
Estimating the 3D pose of the object by comparing features in the 2D input image pairs with features in each 2D reference image pair using a cost function that matches the features;
Including a method.   The method of claim 1, wherein the 2D input image is obtained from a single image acquired by a camera having a non-linear intensity response.   The method according to claim 1, wherein the 3D pose is defined by a 3D translation vector (X, Y, Z) and a 3D Euler angle (μ, φ, σ) with respect to orientation.   The method of claim 1, further comprising refining the pose using a precision cost function.   The method of claim 1, wherein the feature is obtained by image processing intensity resulting from specular reflection.   The method of claim 5, wherein the reference specular reflection intensity is rendered by using a mirror's bidirectional reflection distribution function (BRDF) or some other known BRDF. The method
Placing a mirror sphere in the environment;
Obtaining an environment map image via reflection of the environment around the mirror sphere, and constructing an environment map from the environment map image using a 2D plenoptic function, the 2D A plenoptic function models the surrounding appearance, and the reference image is rendered from a 3D model of the object reflecting the environment map;
The method of claim 5, further comprising:   8. The method of claim 7, further comprising obtaining an image set of the environment and generating a mosaic from the image set to construct the environment map.   The method of claim 1, wherein the amount of exposure used during acquisition of the input image is different.   The short exposure is about 1/60 seconds, the long exposure is about 1/4 seconds, and the camera aperture is adjusted to the ambient illuminance, thereby creating an image with standard intensity by long exposure. The method according to claim 9. The posture is obtained by solving the following equation:

However,

Represents the translation and Euler angles of the initial pose, C _R () is the approximate cost function, I _L and R ^L are the long exposure input image and the long exposure reference image, respectively, I _S and R ^S are 6. The method of claim 5, wherein the method is a short exposure input image and a short exposure reference image, the argument min is a function that returns an argument giving a minimum value, and the inner minimum value is obtained before the outer minimum value. . The approximate function is:

12. The method according to claim 11, wherein λ is a control parameter, and C ₁ () and C ₂ () are cost functions for the long exposure image and the short exposure image, respectively. A highlight pixel is used for C ₁ (), which is obtained by performing a binarization process to create a corresponding binary image, the method comprising:
By applying a distance transform to the binary image, further comprising constructing the corresponding reference distance image D _R and input distance image D _I, The method of claim 12. The cost function C ₁ () is

14, wherein (x, y) is a projection point, (u, v) is a pixel coordinate, N _highlight represents the number of pixels for summation, and S represents short-time exposure. Method. The cost function C ₂ () is

13. The method of claim 12, wherein NCC represents normalized cross-correlation and L represents long exposure. (X, Y) represents translation, (μ, φ, σ) represents Euler angle of the precise posture, and the precise cost function is

16. The method according to claim 15, wherein (u, v) are pixel coordinates of the input image I and the reference image R, NCC represents normalized cross-correlation, and L represents long exposure.   The method of claim 1, wherein the feature is a specular flow.   The method of claim 17, wherein the specular flow is due to rotation of the environment about a predetermined viewing direction of a camera that acquires the 2D input image.   The method of claim 17, wherein the specular flow is determined using block matching and a color coded environment map. (X, Y) represents translation, (μ, φ, σ) represents Euler angle of the posture, and the approximate cost function is

Where λ is a control parameter, C ₁ () and C ₂ () are cost functions based on motion division and the specular flow, respectively, and R and I represent the reference image and the input image, respectively. The method of claim 17. Further comprising constructing the size of the specular flow from said binary image and a distance transform is obtained by binarization processing, corresponding reference distance image D _R and input distance image D _I, the cost The function C ₁ () is

Where (x, y) is the projection point, (u, v) is the pixel coordinate, the summation is performed on the motion-divided pixels of the reference image R, and N _motion represents the number of such pixels. Item 21. The method according to Item 20. Pixels that are not abnormal values in which the difference between the input specular reflection flow vector and the reference specular reflection flow vector is less than a small threshold value by comparing the reference specular reflection flow image R and the input specular reflection flow image I. The cost function C ₂ () is:

21. The method of claim 20, wherein M is a set of pixels that are not outliers. (X, Y) represents translation, (μ, φ, σ) represents the Euler angle of the 3D pose, and the precise cost function is

Where (u, v) is pixel coordinates, R is the reference image, has posture parameters (θ, φ, σ, X, Y), N _mask represents the number of stencils, and the stencil represents the object The method of claim 17, defined as a segmentation mask.   The method of claim 18, wherein the rotation is about ± 5 degrees.   The method of claim 1, wherein each 2D input image pair is generated from a single high dynamic range image.   The method of claim 1, wherein each pair of 2D input image and 2D reference image is generated from a set of images collected using different exposures.   The method of claim 1, further comprising removing the object from a container using a robotic arm according to the estimated posture.   28. The method of claim 27, wherein the container comprises a single object or multiple objects.   The method of claim 1, wherein the 3D pose has six degrees of freedom.   The method of claim 1, further comprising segmenting the object in the input image while estimating the pose.   The method of claim 1, further comprising estimating a reflectance of the object in the input image while estimating the pose.   The method of claim 1, wherein the input image is obtained from multiple views of the object.   The method of claim 1, further comprising actively illuminating the scene with an illumination source.   The method of claim 1, further comprising actively irradiating the environment with an irradiation source.   34. The method of claim 33, wherein the illumination source includes one or more projectors.   The method of claim 1, wherein the input image is acquired using polarized light to estimate a specular reflection component.   The method of claim 1, further comprising illuminating the scene with a plurality of different colors and performing the method independently for each color.   The method of claim 6, wherein the reflectivity is mirror-like.   The method of claim 6, wherein the other known BRDF is that of the object.   28. The method of claim 27, wherein the input image is acquired by a camera mounted on the robot arm.   28. The method of claim 27, wherein the container comprises active illumination. An apparatus for estimating a 3D pose of a 3D object in an environment, the object having a specular reflective surface, the apparatus comprising:
A rendering engine configured to render a set of 2D reference image pairs using a 3D model of the object and a set of poses of the object, each reference image pair associated with one of the poses A rendering engine,
A camera configured to acquire a 2D input image pair of the object;
Means for estimating the 3D pose of the object implemented in a processor and comparing the features in the 2D input image pair with the features in each 2D reference image pair using a cost function that matches the features; ,
An apparatus comprising:

Images