DE102007028735A1 - Method for form and texture collection of object by which three dimensional sensor and digital camera are used for recording of digital photos, involves surrounding object by housing which encloses object with multiple complete solid angle - Google Patents

Abstract

The method involves surrounding an object by a housing, which encloses the object with a multiple complete solid angle. The interior or exterior is lit up directly or indirectly with multiple sources of light so that the interior is illuminated, by which the bright interior serves as source of light for a photo admission with the digital camera. The photos are projected mathematically from the digital photo camera on the 3 dimensional data of the object. An independent claim is also included for device for the form and texture collection of an object.

Description

Technical task and state of the art

In many applications in technology, virtual reality and Preservation become 3D sensors for three-dimensional acquisition the form used. Exemplary tasks are: Capture of consumer goods, measurement of statues. exemplary Measuring principles for 3D optical sensors are laser triangulation or coded lighting, e.g. B. by fringe projection.

The Quality of 3D optical measurement is from the surface of the measured object. Reflective, very dark or transparent surfaces can not usually be measured. Here one often helps oneself by the so-called "dissecting" the object surface. Among them is usually a spraying and thus covering the object surface with a bright, well-scattering material, eg. B. chalk powder or Titanium dioxide, understood.

The resulting 3D data is usually through triangular meshes represents.

Often In addition to the shape of the object and the texture the object surface are detected. This is special asked in the field of virtual reality.

It is known, a so-called "mapping of a texture" on Perform 3D triangular meshes.

The use of textures obtained by digital photography is exemplified in:
Alshawabkeh, Yahya, "Integration of Laser Scanning and Photogrammetry for Heritage Documentation", Dissertation at the Institute of Photogrammetry, University of Stuttgart 2006, pages 66-71 ;
accessible under http://elib.uni-stuttgart.de/opus/volltexte/2006/2899/

To First, the 3D data of the object are measured with a 3D sensor measured. In addition, the system becomes a photographic Camera added, which is equipped with an electronic image sensor is that can take color pictures. The camera becomes relative to Sensor using the method of "camera calibration" known from photogrammetry calibrated The rigid unit of 3D sensor and camera is below referred to as "recording unit".

One optical sensor performs the detection or measurement in the Usually only from one line of sight. A common task is the all-around measurement of the object. To surround the object measured, carried out several shots, between each of which directed to the object receiving unit to various locations around the Object is brought around. Alternatively, the object can also be rotated become. The calculation and evaluation unit controls with the help of a suitable control electronics both the recording unit and optionally a positioning unit for rotating the object.

Then the composing and triangulating of the 3D-recordings and the mapping of the texture, where from the different directions, in which the photographic camera was, the texture data mathematically with the help of a computing and evaluation unit on the 3D data is projected. The parameters of the projection are through set the camera calibration. The 3D data is usually represented as a triangle mesh. At the projection will on each triangle a triangular section of the photographic Recording projected. This triangular section is always used in visualizations treated as if it were firmly on the triangle, so a triangular Detail of the colored surface of the object.

1 shows a photo mapping across two triangles.

The Shooting (3D and photo) is generally done from multiple directions. It is possible in principle, any of the photos from its recording direction to project the 3D data. by virtue of According to the laws of geometry, the texture information is assembled correctly and it creates a completely measured object with texture. requirement Among other things: (a) error-free 3D location information (b) error-free Camera calibration, but also, as in the next section displayed, the same lighting conditions, independent from the direction of illumination.

Criticism of the state of the art

at The implementation of such a system has problems with the transition between the photos taken from different directions.

To the One is the assumptions (a) and (b) not exactly, but in practice only met with a certain error. Thereby When viewing the textured grid, 3D image jumps occur at the crossing points between the projections of the various Photos.

To the others will jump in brightness at the same place observed.

Further, dissection of objects is often not possible, because through the preparation the texture covered and thus can not be photographically recorded.

Objectives of the invention

The Invention is intended to solve the problems described in the prior art to solve. Furthermore, the invention should make it possible for transparent or objects with a reflective component the otherwise diffuse reflection of the triangular networks with an approximate provided texture determined.

Description of the invention

The Jumps in brightness are explained as follows: When recording, there is a certain lighting situation in the Space, z. B. the cloudy sky illuminates the object a window. An object detail scatters the received illumination in the room. At some point in the room is the Photo camera. The scattering is generally not uniform strong in all directions.

The Scattering is almost always strong from the angle between the direction of illumination and the orientation of the object detail. These Dependency ("object function") can be, for example for "ideal scattered objects" by the Lambertsche Law will be described. For real scattered objects is the Dependence on the illumination angle by another function to describe. This other object function is from the surface dependent on the object; for objects in which the light partially can penetrate, moreover, from the scattering properties of the object below its surface. In the human face for example, it is such an object into which the Light partially penetrates. The described surface and scattering properties of skin and facial tissue vary location dependent over the face and are from Different from person to person. But also with other (technical) Objects is the description of the surface and scattering properties complicated. Theoretical models exist, but they are not usable in practice because of ignorance of the parameters.

Tries, Although modeling the object function improves the result the direct mapping of the photos a bit, but the results are still bad and there are still jumps in brightness at the crossing points between the projections of the various Photos. The problem can not be solved without sacrificing quality solved by calculations in the computing and evaluation rather, you have to physically make the meter yourself change.

The problem can be solved according to the invention by an illumination, which arrives equally bright on all objects of the spatial directions (4π). An object in an integrating sphere solves the problem. Thus, according to the invention, in a method or a device, the object ( 4 ) with a housing ( 5 ), which on its inside should be white and well dispersing ( 2 ).

The necessary executions ( 6 ) (Openings) to the optical components of the recording unit ( 3 ) should only claim a small solid angle.

The walls can be semi-transparent and externally large, with suitable lighting ( 8th ). However, this illumination would interfere with the 3D measurement and is preferably switched off for the 3D measurement. You can use flashlights to capture the texture that flash in sync with the photo.

In order to ensure the rotation of the object in the housing for the various shots, a turntable ( 7 ) be used. This should then also reflect white diffuse. In general, instead of the turntable any other positioning occur, for. B. a robot arm.

The approached positions of the turntable or the positioning should preferably for later evaluation be known. If they are not known, algorithms can calculate the transformations from the 3D part views can (so-called registration algorithms). Because that's exactly what is technically possible, are also Ausfürhungsformen possible without turntable, where the object is not in exactly known manner manually rotated and moved. Further is a combination of defined turntable steps and twists possible by hand to capture the object from all sides to be able to. In combination, the registration algorithms need only used for the movements where twists were done by hand. The remaining transformations are calculated on the basis of the known turntable positions.

in the Housing will be multiple scatters on the walls take place so that an approximately homogeneous illumination, evenly setting for all directions thus and also the turntable an approximately the same Brightness gets and even evenly bright appears.

An economical solution for the realization of the lighting of the housing is in 3 outlined:
The walls of the housing ( 5 ) are white, scattering but not transparent.

As image field background ( 10 ) will be the cutout on the in 3 called the right wall of the housing, which corresponds to the field of view of the camera.

Around this image field background ( 10 ) around it is a circumferential gap ( 14 ), through which the light of a flash lamp ( 11 ) which can occur at a suitable scattering surface, preferably with the aid of a "soft box" ( 12 ), scattered. Because the gap is just outside the field of view, the light must undergo some scattering processes until it can enter the field of view. This process homogenizes the lighting. Additional foreclosures ( 9 ) are necessary to prevent a direct Refexion of coming out of the circumferential gap bright light on the measurement object.

The measurement with the sensor can advantageously be from the top in the downward direction ( 15 ), because then in combination with the rotation of the turntable, a measurement of the object can be made from above and from the sides.

In order to be able to measure from below, a second sensor can be used, which measures from below in upward direction. Alternatively, in another embodiment of the invention ( 4 ) a mirror share the image field of the recording unit and as a result of the mirroring the recording unit virtually with a part of its image field from the top downwards inclined direction ( 15 ) and with the other part from with an upward sloping direction ( 16 ) measure up.

If the homogenization of the lighting is not completely successful, and further cracks in the texture when composing the shots so you can use algorithms that approximate assume an object function and with the knowledge of the object normal correct the texture from the 3D measurements. An even easier one Solution is smearing the cracks over Low pass filter.

The Problem that due to errors in the 3D measurement or the camera calibration 3D image jumps at the crossing points between the projections of the various Photos occur, according to the invention thereby to be solved that taking pictures in smaller Angular steps performed with correspondingly more photos because the error propagation of 3D measurement errors and errors in the camera calibration, the smaller the relative Rotation angle between the two shots is. In general it is not necessary, also the number of 3D measurements accordingly increase, provided the number of measurements for the 3D all-around survey is sufficient. With the following procedure a preparation of the object becomes possible: first will only take a series of photographs at certain angular positions of the Turntable made.

Then a wall of the housing is opened and the object sprayed with spin finish. While spraying the turntable is rotated so that the object prepared from all sides. The object may neither be from the Turntable still taken in another way on the turntable to be moved. Then a series of 3D measurements in turntable positions, the one known location to the turntable positions of the photo series to have. Preferably, the same turntable positions are like used in the photo series.

When last procedural steps are the 3D data evaluated and the 3D object calculated, then the photo textures are projected on.

A Preparation is necessary for many objects, in particular in translucent, specular and black objects. Even if the object is measurable three-dimensionally, it improves dissection the accuracy of the measurement results.

In order to the preparation or rotation of the measurement object by hand can take place, can mauellen engagement in the wall of the housing a door can be provided.

With of the following embodiment of the invention transparent objects are visualized. As an example for explanation in this description is an empty, round in supervision round bottle serve partially transparent-blue material. The background color for the intended visualization must be known: B. should the Bottle in a 3D animation against a green background being represented.

To measure must the necessary hardware after 2 to be extended: The image field background ( 10 ) is to be replaced according to the invention by a plate exactly that color in which the visualization is planned later.

If now the measurement proceeds according to the method described, The background appears through transparent and transparent object areas is mapped to the texture, as if the surface had teiweise the color of the background. If the 3D visualization later takes place before the same color, the representation looks real, too if you turn the object virtually.

With the following other embodiment of the invention, the transparency of partially transparent objects can be approximately measured and represented be put.

to Depicting a transparent surface will be common the (R, G, B, α) -model used, wherein R, G, B, the red, green or blue color values are and α is one Absorption value. According to the model, α = 0 means a complete one transparent point and α = 1 a complete opaque point.

According to the invention the approximation is made, the object exists from a partially transparent shell, by partially transparent Dreicksnetze is displayed. The inside of the shell is in the calculation model empty.

When Example for explanation in this description is also intended an empty, in supervision round bottle from teiltransparent-blue Serve material. The background color for the intended Visualization does not need to be known

The necessary hardware also has to be used for the measurement 2 To be extended: According to the invention, a black plate and a white plate should be introduced successively into the field background. Each plate records a series of photos.

At the places where the pictures with white background plate brighter appear, the object was not completely opaque, but partially transparent.

The observed absorption of the object results from the local Ratio β of the brightness of the picture elements of the picture with white background and the picture with black background.

These Absorption is then mapped to the thickening, according to the invention with each half effect on the front of the object located in the projection direction and object back, since the corresponding light beam both Has passed areas.

Appropriate becomes the α-value of the front of the object and the back of the object assigned the root of β, so that the absorption α is visualized, when visualizing a virtual light beam both surfaces happens.

Of the RGB object texture results on the shots with the black Plate in the background. This is because of the translucency of the black background by the factor β too dark. Consequently For example, the RGB values measured with black plate must be divided by β will get the actual object texture.

The Both previously illustrated embodiments will be weaknesses exhibit in the visualization when the object is not inside is empty. The difficulties are compounded when that Object is not rotationally symmetric. As an example, a Serve in plan rectangular bottle with translucent blue Liquid is filled. A recording of a Object orientation, in which the object is the longest for the light rays have to be traversed, becomes stronger absorption have (in the example: in a deeper blue) as a recording with a shorter distance for the rays of light. When assembling are at the crossing points between the projections of the various photos jumps in the Depth of staining due to different lengths of absorption expected.

The Problem is inventively by the reduction the angular steps of the turntable with simultaneous increase solved the number of shots, because the smaller the angular step, the smaller the difference in the described path lengths. Thus, the jumps described in the brightnesses smaller. For carrying out the method, according to experience, angle steps of 15 ° or less necessary. Thus, at least 24 Photographs necessary at one turn of the turntable.

The three previously illustrated embodiments will be weaknesses in the visualization, if the object is not only partially transparent is, but in its interior also scatters the light. As an an example should serve a reckteckige in supervision bottle, which with a slightly scattered cloudy liquid filled is. The background color for the visualization is green.

A Recording in an object orientation in which the object is the longest for the light rays to be traversed route is have greater scattering (in the example: white-green with a high proportion of white appear) as a recording with a longer Range for the rays of light. (there would be white-green to be expected with a high proportion of green). When assembling are at the crossing points between the projections the various photos jumps in the depth of the coloring due to different lengths of absorption to expect.

According to the invention the problem through the use of a white background at the data intake and a white background at the Visualization solved, because the observed (more or less strong) whitening stems from the white case color outside the image field of the Camera here. By doing so, the two color components, which mix in different proportions, same color.

1

photo of the measurement object

2

triangle mesh

3

recording unit

4

measurement object

5

white case 6 : Feedthroughs to the receiving unit

7

turntable

8th

lighting

9

isolation

10

Field background

11

Speedlight

12

softbox

13

mirror

14

surrounding gap

15

direction of observation inclined downwards from above

16

direction of observation tilted from below upwards

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited non-patent literature

• - Alshawabkeh, Yahya, "Integration of Laser Scanning and Photogrammetry for Heritage Documentation", Dissertation at the Institute of Photogrammetry, University of Stuttgart 2006, pages 66-71 [0006]
• - http://elib.uni-stuttgart.de/opus/volltexte/2006/2899/ [0006]

Claims (9)

Method for shape and texture detection of an object, in which a 3D sensor and a digital camera are used for recording digital photos, characterized in that - the object is surrounded by a housing which - the object with a large part of the complete solid angle (4π) encloses and - whose inside or outside is illuminated directly or indirectly with one or more light sources so that the inside lights itself and - in which the luminous inside serves as a light source for taking a photo with the digital camera and - the object inside the housing with the help a positioning unit or manually placed in different positions - a sequence of photographs taken in different positions of the object, which allows a texture measurement from different sides - a sequence of 3D images taken in the different positions of the object, which is a 3D measurement of different Pages and - the digital camera photos are mathematically projected onto the 3D data of the object. Method according to claim 1, characterized in that that in a further process step after recording the Taking photos and before recording the 3D shots a preparation of the measured object takes place. Device for determining the shape and texture of an object, consisting from a 3D sensor and a digital camera characterized that - The object surrounded by a housing is which - the object with a large part of the complete solid angle (4π) encloses and - its inside or outside directly or illuminated indirectly with one or more light sources so that the inside itself shines and - in which the glowing inside as a light source for a photo shoot with the digital camera serves and A positioning unit, with the help of which the object inside the case in different Positions can be brought - And a control electronics and a computing and evaluation unit, which is a sequence of photographs and control 3D shots in different positions of the object can and thus enables a measurement from different sides Device for determining the shape and texture of an object, consisting from a 3D sensor and a digital camera characterized that - The object surrounded by a housing is which - the object with a large part of the complete solid angle (4π) encloses and - its inside or outside directly or illuminated indirectly with one or more light sources so that the inside itself shines and - in which the glowing inside as a light source for a photo shoot with the digital camera serves and - a door in the housing through which, when opened, the operator manually moves the measurement object to different positions can - And a control electronics and a computing and Evaluation unit, which a sequence of photos and 3D shots in different positions of the object can drive and in this way allows a measurement from different sides Method according to claim 1 or 2, characterized that a mirror divides the image field of the recording unit, wherein the Object both in the remaining part of the direct image field as well can be measured in the mirrored part of the image field. Method according to claim 1, 2 or 5 for detection and visualization of transparent objects, characterized that in the image field of the photo camera behind the measurement object during the process a photo background is introduced, which is the color the background of the later visualization having. Method according to claim 1, 2 or 5 for detection and visualization of transparent objects, characterized, that - in a procedural step immediately after the recording of the photos in the field of view of the camera behind the measurement object is a black photo background is introduced and - in a further step additional photo shoots done with a black photo background in the frame of the camera, which correspond to those with the white background. - that the absorption in the object point from the root of the local quotient β of Brightness of the pixels with black and white background is calculated and - that the object color value from the Object color value of pixels with black background divided by This local quotient β is calculated. The method of claim 1, 2, 5, 6 or 7 for detection and visualization of transparent, volume scattering objects, characterized in that in the field of view of the camera behind the measurement object a white photo background is inserted, and the later Visualization done with a white screen background. The method of claim 1, 2, 5, 6, 7 or 8, characterized marked that the sequence of photographs at least 24 Images includes.