DE102017100481A1 - SCALING A GEOMETRIC MODEL OF A MEASUREMENT AREA DETECTED BY AN IMAGE OF IMAGE DATA - Google Patents

Abstract

Shown is a system for scaling a geometric model of a measuring range. The system includes an image pickup device having a first image pickup unit disposed at a first position in a first orientation and a second image pickup unit disposed at a second position in a second orientation, and an evaluation device, wherein the first position, the first position Alignment, the second position and the second orientation do not allow simultaneous recording of part or all of the measuring range. The evaluation device is set up to determine data relating to a movement of the image recording device from a series of images of the first image recording unit and / or the second image recording unit and using at least one image of the first image recording device and at least one image of the second image recording device, which images the measuring region and to scale the geometrical model of the measurement area, in particular metric, taking into account data relating to a relative position and a relative orientation between the first image acquisition unit and the second image acquisition unit.

Description

TERRITORY

The present invention relates to the scaling of a geometric model. More particularly, the present invention relates to the scaling of a geometric model of a measurement range for determining lengths.

BACKGROUND

To generate a geometric model of an object, it is known from the prior art to use stereo camera systems which simultaneously image the object from different perspectives and determine a three-dimensional position of object points by means of triangulation or triangulation-based methods.

The use of stereo camera systems is advantageous in that there are constant and accurate positional relationships between the cameras that can be used directly for the triangulation calculation. The provision of stereo camera systems, however, is associated with not inconsiderable expense.

SUMMARY

It is therefore the object of the present invention to reduce the effort involved in creating an image-data-based scaled geometric model of a measuring range. This object is achieved by a system according to the invention and a method for using the system.

The inventive system for scaling a geometric model of a measuring area comprises an image recording device with a first image recording unit, which is arranged at a first position in a first orientation, and a second image recording unit, which is arranged at a second position in a second orientation, and an evaluation device wherein the first position, the first orientation, the second position and the second orientation do not allow simultaneous recording of a part or of the entire measuring range and wherein the evaluation device is adapted to collect data from a series of images of the first image recording unit and / or the second image recording unit with respect to a movement of the image pickup device and using at least one image of the first image pickup device and at least one image of the second image pickup device, which map the measurement region, and taking into account ng of data relative to a relative position and a relative orientation between the first image pickup unit and the second image pickup unit to scale the geometric model of the measuring range, in particular metric.

The term "scaling", as used in the description and claims, is to be understood in particular as the assignment of a scale to the geometric model. By way of example, it is possible by means of the scale to assign a physical path to two points in the geometric model which, within the scope of the measurement accuracy achieved, corresponds to the physical distance between corresponding points of the measuring range. Further, the term "geometric model" as used in the specification and claims particularly includes a set of points and / or geometric (fundamental) bodies (eg, lines, areas, spheres, cylinders, cuboids, etc.). ), their relative position and / or orientation in a coordinate system by position values (for example Cartesian coordinates) and / or alignment values (for example values of a rotation matrix, values of a quaternion representation, values of an Euler-Rodrigues representation, etc.). is specified.

Furthermore, the term "measuring range" as used in the description and the claims is to be understood in particular as meaning a volume in which one or more surfaces of one or more spatial bodies are arranged, which are represented by the geometric model in virtual form (FIG. for example, in the form of a network describing the surface (s)), the virtual shape in particular being able to be chosen so as to allow it to derive dimensional information corresponding to the extent of the spatial body or bodies. Furthermore, the term "image recording device", as used in the description and claims, is to be understood in particular as a camera which is set up image and / or video data (in particular image data supplemented and / or differentially encoded with further data typically constant frame rate). In addition, the term "evaluation device" as used in the description and the claims is to be understood in particular as a processor that is set up to process image and / or video data in accordance with instructions provided and to generate output data accordingly. Finally, the phrase that the image pickup units "do not allow simultaneous recording of part or all of the measurement range" as used in the specification and claims means, in particular, a (sometimes finely adjustable) relative orientation of the image pickup units that are not overlapping Image area allows.

The system according to the invention is thus arranged, offset in time, but in temporally related images of the measuring range to derive a scale of the geometric model from the comparison of distances in different reference systems. Possible reference systems in this context are, for example, an (unscaled) coordinate system describing the (relative) movements (in relation to which the description of the geometric model is present) and a (scaled) coordinate system describing the relative position of the image recording units. The scaling is based, for example, on data on the (unscaled) relative movement in relation to the scaled relative position and relative orientation of the image recording units during the process. Thus, by assigning features in the images of the measuring area of the image recording units, taking into account the movement of the image recording units, physical distances can be assigned to distance units in the geometric model.

In this context, it should be noted that features may include regions with particularly large contrast values, different color information, geometries that are particularly different from the environment, such as corners or circles, or edges derived from structural changes in the physical scene. The missing overlap of the image areas is thus compensated by a movement of the image acquisition units and a tracking of the movement by one or both image acquisition units, so that, in simple terms, a scalable database can be generated from the temporally and spatially related double monocular process.

Preferably, the evaluation device is further configured to provide one or more lengths extracted from the scaled geometric model.

The scaled geometric model can thus be used to measure distances in the measuring range.

Preferably, the evaluation device is further configured to receive a selection regarding the one or more lengths extracted from the scaled geometric model.

The selection may, for example, consist in marking points or one or more objects in images of the measurement area and, in response, providing a distance between the points or dimensions of the one or more objects through the system.

Preferably, the selection comprises a categorization of an object to be measured or an area to be measured within the measuring area, and the evaluation device is further configured to provide, based on the categorization, a measuring instruction associated with the object to be measured or the area to be measured.

In this case, the term "measuring instructions" as used in the description and the claims, in particular a graphical (eg text-based or animated), acoustic (eg linguistic), and / or haptic-based (eg by means of vibration) description to understand steps to be taken in the measurement. The measurement instructions may include, for example, a determination of one or more measurement areas and information / graphics regarding points or one or more objects to be marked in the one or more measurement areas and / or the time sequence of the points to be marked or one or more objects.

Preferably, the evaluation device is further configured to receive a categorization of an object to be measured or a region to be measured within the measuring range and to generate or update a measuring instruction assigned to the category on the basis of the selection.

The measurement instructions assigned to a category may include, for example, typical measurement points or measurement sections for objects or regions of the category. For example. For example, a measuring instruction for a person to determine a size of clothing may contain, in addition to the measures to be determined, a guide in which area, for example, a waist circumference is to be measured. Furthermore, measuring instructions for determining an allowance, depending on the type of construction work (category) desired, may include data on which routes are to be measured. Furthermore, a measuring instruction can be created or updated by evaluating a measuring process, for example by storing markings of a user and / or comments on the markings in connection with a category selected or created by the user. Measuring processes can be compared with several users in order to generate or update measurement instructions.

The first image recording unit is preferably arranged on a first side of the image recording device and the second image recording unit on a second side of the image recording device that is parallel to the first side.

Said arrangement is in many mobile devices such as mobile phones, tablets, Laptops, etc. common because said devices often have a first camera on the screen side for performing video telephony and a second camera on the opposite side to make pictures or videos and can therefore be realized with little effort. Furthermore, existing devices can be retrofitted or expanded without much effort (eg. By means of software or calling an accordingly programmed website).

Preferably, the image pickup device further comprises an input device and a screen.

The input device and the screen simplify the selection, for example, the marking of points or of one or more objects in images of the measuring range as well as the display of the measuring instructions or the measured lengths.

The image recording device preferably comprises the evaluation device or the image recording device is configured to transmit the images to the evaluation device via a network, in particular by means of a radio link.

For example. the evaluation device can be realized by means of a software (app) that is executed by a processor on a mobile terminal such as a mobile phone, a tablet, a laptop, etc., or an app on a mobile phone, a tablet, a laptop , etc. transmit the images to a server which performs the necessary calculations and provides the results to the mobile terminal for display. Furthermore, the server can store the transferred images and calculations as a backup copy or for later processing, possibly using another computer.

Preferably, the first position, the first orientation, the second position and the second orientation are fixed or the first image recording unit and the second image recording unit are coupled.

In this context, "firm" means a permanently rigid (albeit possibly detachable) connection between the first image recording unit and the second image recording unit, which does not allow any change in the relative position and orientation, for example a fixation in a rigid housing which supports the Einhaust system or said system components einhaust. Furthermore, the term "coupled", as used in the description and the claims, in particular an adjustable or flexible connection to understand. While fixed connections do not require a continuous determination of the relative position and orientation of the image recording units, this is necessary in the case of coupled image recording units if there is a change in the relative position and orientation of the image recording units during the process. Furthermore, even with a temporarily "fixed" coupling, a current relative position and orientation of the image recording units must be available.

Preferably, the geometric model is a three-dimensional model.

As a result, different views can be generated from the model and lengths can be extracted.

Preferably, a method of scaling lengths within a measurement area using the system comprises mapping the measurement area with the first image acquisition unit, rotating or panning the image capture device, and mapping the measurement area with the second image acquisition unit, the series of images during rotation or panning the image pickup device and the series of images consists of at least partially overlapping images.

Thus, from the overlapping images, the movement of the image pickup device or the first image pickup unit and / or the second image pickup unit can be determined. Alternatively, the determination of the movement may be supported by measurement data from inertial sensors, or the motion in the case of image sequences comprising non-overlapping images may be partially estimated from the measurement data of the inertial sensors.

The determination of the data with regard to the movement of the image recording device during the rotation or pivoting of the image recording device by the evaluation device preferably comprises an assignment of features detected in image pairs.

Due to the correlation of features, a relative movement of the image recording device or the first image recording unit and / or the second image recording unit between image pairs can be determined.

Preferably, the scaling of the model of the measurement area comprises associating features detected in image pairs of the series of images, the image pairs each comprising an image of the first image acquisition unit and an image of the second image acquisition unit.

By associating images of the first image acquisition unit and the second image acquisition unit, a compensation calculation can mediate Scaling observations are made based on the existing overdetermined observations of correlating features between the first image acquisition unit and the second image acquisition unit over a subset or all image pairs.

In particular, when the individual images of the own measurement series of the first image recording unit or the second image recording unit allow an annular end of the measuring range, for example by a rotary or pivoting movement about its own axis by 360 °, there is a redundancy, inter alia by the Adjustment of mediating observations can be used to improve the accuracy of the geometric model.

Preferably, the method further comprises displaying information about a desired motion guidance with respect to the rotation or pivoting of the image sensing device on a screen.

In this case, the desired movement guidance may, for example, be selected such that a plurality of separate measurement areas or an annular measurement area are imaged by both image acquisition units in the context of a rotary or pivoting movement.

Preferably, the information is continuously updated based on an evaluation of the series of images during rotation or panning and, in particular, includes information regarding the direction and / or the travel length of rotary or panning motion sections.

For example. an evaluation of the images during rotation or pivoting can take place and in the case of non-overlapping or insufficiently overlapping images of the same image recording unit or of the two image recording units, a repetition of the movement can be demanded or displayed.

list of figures

• 1 eine schematische Darstellung eines beispielhaften Systems;
• 2 eine schematische Darstellung hinsichtlich einer beispielhaften Bewegung einer Bildaufnahmeeinrichtung während eines Prozesses zum Skalieren eines geometrischen Modells eines Messbereichs; und
• 3 ein Flussdiagramm eines beispielhaften Prozesses zum Skalieren eines geometrischen Modells eines Messbereichs.

The invention will be explained in more detail in the detailed description on the basis of exemplary embodiments, reference being made to drawings in which:

• 1 a schematic representation of an exemplary system;
• 2 a schematic representation of an exemplary movement of an image pickup device during a process for scaling a geometric model of a measuring range; and
• 3 a flowchart of an exemplary process for scaling a geometric model of a measuring range.

The same or analogous elements are indicated by identical reference numerals in the drawings.

DETAILED DESCRIPTION

1 shows a schematic representation of an exemplary system 100. The system 100 includes a first image pickup unit 210 (For example, a first camera), which on a front side (in 1 left) of an image pickup device 200 is arranged, and a second image pickup unit 220 (For example, a second camera), which on a back (in 1 shown on the right) of the image pickup device 200 is arranged. The image capture device 200 may comprise a rigid or flexible housing into which the image-recording units 210 . 220 are integrated and through which the first image acquisition unit 210 and the second image pickup unit 220 rigid or flexible coupled together.

Furthermore, the system includes 100 an evaluation device 300 , configured for evaluating image data, and a touch-sensitive display 400 , set up to make input and display expenses. Image recording device 200 , Evaluation device 300 and touch-sensitive display 400 can, as in 1 illustrated, in a device (eg. A mobile phone, a tablet, a laptop, etc.) to be integrated. Alternatively, the image pickup device 200 and the evaluation device 300 be integrated into different devices that are connected, for example, by means of a network connection. For example. can the evaluation device 300 be integrated into a server, the image data from the image capture device 200 receives and evaluates.

2 and 3 illustrate an example process for scaling a geometric model of a measurement area 500 , It will be from the first image acquisition unit 210 (whose position and orientation in 2 indicated by dashed lines) and the second image pickup unit 220 (whose position and orientation in 2 is also indicated by dashed lines) recorded synchronously in time or in a certain temporal relation to each other continuously images, while the image pickup device 200 is moved (panned).

It is sufficient to determine the movement of the image pickup device 200 out, low-structure objects 600 For example, tables, walls, chairs, etc. are extracted from the images. For example, features (lines, dots) are extracted that make up the movement of the image recording device 200 between two consecutive pictures of the same picture-taking unit 210 respectively. 220 can be calculated. The calculation of the movement can also be supported by additional sensors, such as inertial sensors (eg acceleration sensors, rotation rate sensors, etc.) or bridged for a short time (with insufficient image material).

Furthermore, a (feedback regarding a) motion specification to the user can be made by motion analysis and comparison to a simulated motion using the image data and (if available) the sensor data of the additional sensors. For example. can, as in 2 indicated, the second image pickup unit 220 continuously objects 600 from which the relative movement of the image recording device 200 can be calculated. For example, the position and orientation of the image pickup device 200 by the "perspective n-point method" (see, for example, US Pat. Gao, X.-S., Hou, X.-R., & Tang, J., August 4, 2003, Complete solution classification for the perspective-three-point problem, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, S. 930-943, IEEE ), provided that sufficient features are present in the scene or in the room.

Otherwise, the position and orientation of the image pickup device 200 as well as the reconstruction of the measuring range 500 by the "iterative-closest-multiple-lines-method" (see eg. Mentges, G., & Grigat, R.-R., 16-21. May 2016, Surface Reconstruction from Image Space Adjacency of Lines Using Breadth-First Plane Search, INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), IEEE, 2016, p. 995-1002, Stockholm / Sweden: IEEE ) respectively. In addition to the reconstruction of the measuring range 500 By means of the mentioned method by means of lines, a planar reconstruction by the "dot-based dense method" (see, for example, FIG. Marcinczak, JM, & Grigat, R.-R., September 2014, Total Variation Based 3D Reconstruction from Monocular Laparoscopic Sequences, MICCAI 2014 WORKSHOP ON ABDOMINAL IMAGING: COMPUTATIONAL AND CLINICAL APPLICATIONS, Boston: Lecture Note in Computer Science (LNCS) Volume 8676 ) be performed.

On the basis of all features, a compensation calculation can also be used for mediating observations, inter alia with regard to the positions and orientations of the image recording device 200 be performed. The movement of the image pickup device 200 is there, as in 2 shown to continue until each section of the entire area to be measured 500 on at least two images of a series of images of the image acquisition unit 210 or the image capture unit 220 is shown, for example. By a semi-circular motion is performed.

In addition, it is necessary for the scaling estimate that the field of view of the first image acquisition unit 210 at the starting point with the field of view of the second image recording unit 220 overlaps at the end point in a semicircular movement, wherein the measuring range or a part of the measuring range lies in the overlapping viewing area. Furthermore, for accuracy optimization, the measuring range can be extended, for example by performing a complete circular movement, so that each image of the first image recording unit 210 an overlapping viewing area with one or more images of the second imaging unit 220 having.

Due to the overlapping viewing areas of the images of the image recording units 210 with those of the image acquisition unit 220 For example, the respective relative movements can be converted into a coordinate system so that the information about the rigid couplings can be present in the model as well as in relation to a physical variable redundantly as a function of the number of points of view and can thus be compensated.

On the basis of the points generated, it is also possible to extract a surface model which is approximated or generalized, for example, by rule geometries (for example lines, surfaces, spheres, cylinders, cuboids, etc.) as a function of the field of application, so that metric relationships among the objects can be made as distance, shape or position. After saving the measurements and the measurement context such as, for example, a category (for example "measuring a cabinet") in a database, the database can be evaluated for various relevant patterns (for example "user XY has measured a cabinet in this way"), to generate or update measurement instructions. Said measurement instructions can then be made available to the user and, for example, on the touch-sensitive display 400 be presented as a motion preset.

LIST OF REFERENCE NUMBERS

100

system

200

Image recording device

210

Imaging unit

220

Imaging unit

300

evaluation

400

Touch-sensitive display

500

measuring range

600

objects

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• Gao, X.-S., Hou, X.-R., & Tang, J., August 4, 2003, Complete solution classification for the perspective-three-point problem, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, S. 930-943, IEEE [0045]
• Mentges, G., & Grigat, R.-R., 16-21. May 2016, Surface Reconstruction from Image Space Adjacency of Lines Using Breadth-First Plane Search, INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), IEEE, 2016, p. 995-1002, Stockholm / Sweden: IEEE [0046]
• Marcinczak, JM, & Grigat, R.-R., September 2014, Total Variation Based 3D Reconstruction from Monocular Laparoscopic Sequences, MICCAI 2014 WORKSHOP ON ABDOMINAL IMAGING: COMPUTATIONAL AND CLINICAL APPLICATIONS, Boston: Lecture Note in Computer Science (LNCS) Volume 8676 [0046]

Claims (15)

A system for scaling a geometric model of a measurement area, comprising: an image recording device with a first image pickup unit disposed at a first position in a first orientation, and a second image pickup unit disposed at a second position in a second orientation; and an evaluation device; wherein the first position, the first orientation, the second position and the second orientation do not allow simultaneous recording of a part or the entire measuring range, but in a temporal relation to each other; wherein the evaluation device is set up to from a series of images of the first image pickup unit and / or the second image pickup unit to determine data regarding a movement of the image pickup device; and using at least one image of the first image recording device and at least one image of the second image recording device, which map the measuring region, and taking into account data relating to a relative position and a relative orientation between the first image recording unit and the second image recording unit, the geometric model of the measuring region, in particular metric, to scale. System after Claim 1 wherein the evaluation device is further configured to provide one or more lengths extracted from the scaled geometric model. System after Claim 2 wherein the evaluation device is further configured to receive a selection regarding the one or more lengths extracted from the scaled geometric model. System after Claim 3 wherein the selection comprises a categorization of an object to be measured or an area to be measured within the measuring area, and the evaluation device is further adapted to provide, based on the categorization, a measuring instruction associated with the object to be measured or the area to be measured. System after Claim 3 , wherein the evaluation device is further configured to receive a categorization of an object to be measured or a region to be measured within the measuring range and to generate or update a measuring instruction assigned to the category on the basis of the selection. System according to one of Claims 1 to 5 wherein the first image pickup unit is arranged on a first side of the image pickup device and the second image pickup unit on a second side of the image pickup device that is parallel to the first side. System according to one of Claims 1 to 6 wherein the image pickup device further comprises an input device and a screen. System according to one of Claims 1 to 7 , wherein the image recording device comprises the evaluation device; or the image recording device is set up to transmit the images via a network, in particular by means of a radio link, to the evaluation device. System according to one of Claims 1 to 8th wherein the first position, the first orientation, the second position and the second orientation are fixed or the first image recording unit and the second image recording unit are coupled. System according to one of Claims 1 to 9 , where the geometric model is a three-dimensional model. A method of scaling lengths within a measurement range using a system according to any one of Claims 2 to 10 imaging, comprising: mapping the measurement area with the first image acquisition unit; Rotating or pivoting the image pickup device; and imaging the measuring area with the second image pickup unit; wherein the series of images is taken during the rotation or pivoting of the image capture device and the series of images consists of at least partially overlapping images. Method according to Claim 11 wherein the determining of the data with respect to the movement of the image recording device during the rotation or pivoting of the image recording device by the evaluation device comprises an assignment of features detected in image pairs. Method according to Claim 11 or 12 wherein scaling the model of the measurement area comprises associating features detected in image pairs of the series of images, the image pairs each comprising an image of the first image acquisition unit and an image of the second image acquisition unit. Method according to one of Claims 11 to 13 , further comprising: displaying information about a target motion guide regarding the rotation or panning of the image pickup device on a screen. Method according to Claim 14 wherein the information is continually updated based on an evaluation of the series of images during rotation or panning, and in particular includes information regarding the direction and / or travel length of rotary or panning motion sections.

Images