JP2003091716A - Three-dimensional space measurement data accumulating method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional space measurement data accumulating device for operating the intermediate presentation data of a three-dimensional space measured by other three-dimensional measuring method on the basis of the image data of the measured three-dimensional space, and describing and accumulating a result as the tag data. SOLUTION: An intermediate representation data operating means 109 operates the intermediate representation data of the case where the three- dimensional space was measured by other n-kinds (n is positive integer of 1 or more) of three-dimensional measuring methods on the basis of a plurality of image data accumulated in an initial image data accumulating means 103, and the initial tag data accumulated in an initial tag data accumulating means 107. The tag indicating a result of the operation by the intermediate representation data operating means 109 is described by means of an XML language corresponding to the plurality of image data, and accumulated in an after- processing tag data accumulating means 111 as n-kinds of tag datum.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to three-dimensional space measurement data for measuring the three-dimensional shape and image of space and accumulating tag data necessary for applying other measurement methods and data display methods. The present invention relates to a storage method and device.

[0002]

2. Description of the Related Art Displaying an actual room or outdoors in virtual reality or computer graphics is
Therefore, it takes time and effort to input the necessary display data into the computer, resulting in high cost. In order to use these technologies more realistically, a means for automatically inputting three-dimensional data in a wide space is needed.

[0003]

However, although there are various means for measuring a narrow space with high accuracy at present,
There is no general method for integrating these results to create a unified spatial description. Therefore, there is a need for a method that integrates the results of various measuring means into one spatial description.

An object of the present invention is to calculate intermediate representation data of a three-dimensional space measured by another three-dimensional measuring method based on the measured image data of the three-dimensional space and describe the result as tag data. It is to provide a method and an apparatus for accumulating three-dimensional spatial measurement data.

Another object of the present invention is to measure a three-dimensional space from an omnidirectional image measuring device and create a tag description of an image using an extensible description scheme such as the XML language, thereby resulting in other results. Another object of the present invention is to provide a three-dimensional space measurement data storage method and device for storing the description of the three-dimensional space measured by the three-dimensional measurement method as tag data.

[0006]

In the method for accumulating three-dimensional spatial measurement data according to the present invention, the following steps are executed.

In the first step, a plurality of image data are acquired by accumulating in a database by measuring the three-dimensional space to be measured with a three-dimensional space measuring means while changing the viewpoint. As the three-dimensional space measuring means, for example, an omnidirectional image measuring device can be used. By using the omnidirectional image data measured by the omnidirectional image measuring device, the image data in the three-dimensional space can be obtained relatively easily, and the subsequent data processing becomes easy.

Then, a tag indicating the correspondence between a plurality of landmarks respectively included in the plurality of image data measured by the three-dimensional space measuring means is described corresponding to a part or all of the plurality of image data, This is stored in the database as initial tag data. The landmark means a known place (a place where the position can be clearly recognized) included in the plurality of image data, which is defined for specifying the position. For example, a clear predetermined vertical line included in the image can be used as the landmark. An extensible markup language such as the XML language can be used to describe the tag for the image data.

Next, based on the plurality of image data and the initial tag data, the intermediate representation data when the three-dimensional space is measured by one or more other three-dimensional measuring methods is calculated. Other one or more three-dimensional measurement methods include, for example, a multi-view stereoscopic method.

Finally, a tag indicating the result of the operation is described corresponding to a plurality of image data and stored in the database as one or more other tag data. By using the method of the present invention, it is possible to easily execute the processing of a three-dimensional image based on the data measured by another three-dimensional measurement method by using the accumulated plurality of tag data.

A three-dimensional space measurement data accumulating device for implementing the method of the present invention uses a plurality of image data obtained by measuring the three-dimensional space to be measured by the three-dimensional space measuring means while changing the viewpoint. An initial image data storage means for storing as data, a landmark check means for automatically checking the correspondence between a plurality of landmarks included in a plurality of image data, and a tag indicating the check result by the landmark check means are provided. Based on the plurality of image data and the tag data, other n types (n is 1
Intermediate expression data calculation means for calculating intermediate expression data when a three-dimensional space is measured by the above three-dimensional measurement method (positive integer), and a tag indicating the calculation result of the intermediate expression data calculation means for a plurality of image data. It is composed of n post-processing tag / data storage means which are correspondingly described and stored as n types of tag data.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a conceptual diagram used for explaining the concept of the embodiment when the three-dimensional spatial measurement data storage method of the present invention is executed. In FIG. 1, reference numeral 1 denotes an omnidirectional image measuring device used as a three-dimensional space measuring means for measuring the three-dimensional space to be measured by changing the viewpoint. A mobile robot 3 forming a part of the measuring device 1 and an omnidirectional mirror 5 mounted on the mobile robot 3 are illustrated. Mobile robot 3
Is equipped with an omnidirectional image sensor that captures the image transferred to the omnidirectional mirror 5. The omnidirectional image measurement device 1 images the three-dimensional space 7 while moving the mobile robot 3.
As will be described later, a plurality of omnidirectional images (three-dimensional image data) PD1 to PD3 obtained by changing the shooting position are accumulated in the database. FIG. 1 shows a row of omnidirectional images captured at three observed locations. In this example, the vertical line in the subject is the known landmark LM.
Specifically, three or more positions are known landmarks LM.
As the mobile robot 3, while moving the image sensor, a large number of omnidirectional image data whose positions are known within a certain error range on the moving path are collected. For example, 10 cm
Image data is collected while moving the mobile robot 3 at intervals.

In the present invention, the omnidirectional image measuring device 1
Initial tags indicating the corresponding relationships of the plurality of landmarks LM respectively included in the partial areas or all areas in the plurality of image data PD1 to PD3 measured by are described in association with the plurality of image data, and are used as initial tag data. Accumulate in the database. That is, in the image group, corresponding vertical lines (areas), that is, landmarks LM are designated, and they are the same vertical line (landmark), which is one vertical line in the actual three-dimensional space. Is stored as initial tag data. In other words, an image group whose shooting position is known is divided using tags, and initial tag data that describes position information (coordinates) in which the divided vertical lines (areas) exist in three-dimensional space is created. With that, tagging of real three-dimensional space is realized. The block indicated by reference numeral 9 in FIG.
It is a storage means for storing tag data in which tags are described in the XML language. The definition of the image tag in the XML language can be defined as a space description having the following structure.

Multi-viewpoint image: Described as an omnidirectional image sequence whose position is known with a certain degree of accuracy by an omnidirectional image sensor mounted on the mobile robot 3.

Point: A point in the three-dimensional space is designated as (x, y, z) data, and at the same time, the position (x, y) of the point in many omnidirectional images is described.

Line: A line in space is described as a line connecting between two points, and at the same time, it is expressed as a line connecting the positions of both end points in each image. What is actually measured is a line, and points are positioned as their endpoints or intersections.

Surface: A surface is defined as a closed combination of connected lines. However, the shape of the surface is not limited to a flat surface, and the three-dimensional depth of the surface is separately expressed as a depth image, and at the same time, a texture image to be attached to the surface of the depth image is created from the measurement result. These are used not only for three-dimensional curved surface representation but also for display.

With the above arrangement, points, lines, and planes corresponding to each multi-viewpoint image are defined up to the plane, and then three-dimensional coordinates can be automatically calculated.

Solid: Defined as a connection of faces.

Walls: At the initial stage where detailed depth and three-dimensional information cannot be obtained, prominent vertical lines in the environment are measured, and the region between them is represented as a planar wall connecting the vertical lines. The pattern (texture) on the wall can be used as it is as input data for precise multi-view stereoscopic vision.

FIG. 2 is a flow chart showing an algorithm of an example of a method of determining initial tag data. First, in step ST1, a plurality of omnidirectional image data are input.
Then, in step ST2, a landmark LM composed of, for example, a clear vertical line (vertical edge) between the omnidirectional image data strings.
Is detected and each landmark in each image data is associated. Then, in step ST3, a known landmark (LM) is detected within a range that can be discriminated even if there is some error in the position of the robot. Next, the accurate position of the mobile robot 3 is calculated from the orientation of the known landmark LM (step ST4), and the position of the known landmark (LM) is measured again (step ST5). From the accuracy of the position, the correctness of the association in step ST2 can be verified. If there is an erroneous correspondence, the correction is performed (step ST6). Thereby, the position of the robot (that is, the position of each image in space) is accurately measured. Next, in step ST7, the position of the clear vertical line in the three-dimensional environment that has been successfully associated in the series of image data strings is calculated and added as a new landmark LM. As you continue to move, you can see the early known landmark L
M becomes far, and the position measurement becomes inaccurate. Therefore, (1) it is relatively close to the robot, (2) it can be estimated that the number of times of measurement is large and the accuracy of the position is high, and (3) the arrangement of angles is balanced between the known and additional landmarks LM. 3. Find the three combinations (LM set) and thereby perform the position calculation. In that case, there can be a plurality of LM sets, in which case the reliability is weighted from the arrangement of each LM set, and the weighted average value is used as the image position.

If the azimuths of three landmarks in an environment whose position is accurately known are measured, an arc can be written by combining any two of the three landmarks based on the principle of constant circumferential angle. , The intersection of the arcs is obtained. Furthermore, it is possible to select a close one from a plurality of intersections as a measurement position depending on the approximate position of the robot.

By the above calculation, a large number of LMs (vertical lines) in the space are detected from the omnidirectional image sequence measured with the movement of the robot, and the (x, y) coordinates in the space are obtained. This vertical line corresponds to a line radially extending from the center in the omnidirectional image, and the fan-shaped image area between the lines is arranged between the landmarks LM, resulting in a plurality of wall-shaped images. The initial measurement result arranged in the space is obtained. Therefore, the vertical line LM in each image, the correspondence between them, and the position in the three-dimensional space are output as an image tag defined by the XML language, and combined with the original omnidirectional image group to be used as the initial space description. . As a result, one database is formed by the omnidirectional image group and the tag description for the omnidirectional image group, and the three-dimensional space description is generated by tagging the two-dimensional image group.

FIGS. 3A and 3B show examples of omnidirectional images taken at different positions. To facilitate understanding, the positions of the landmarks LM are shown by black lines extending radially in FIGS. 3A and 3B. In addition, FIGS. 4A to 4C are diagrams in which an omnidirectional image is developed in a cylinder, and in this figure, each landmark LM is indicated by a vertical black line. The correspondence between the landmarks LM in each image shown in these figures is stored as tag data. A developed image of an omnidirectional image as shown in FIG. 4 is displayed on the display unit, and, for example, when a person designates a point corresponding to two of them, a line or a plane as a connection, the point is immediately displayed. The device can be configured so that a three-dimensional position is calculated and the position of that point is automatically displayed even in an image that is not designated. By doing so, even if there is a large amount of omnidirectional images, each point needs to be specified only once, and a person can efficiently input tags without being aware of the three-dimensional space or the columns of omnidirectional images.

In the present invention, a plurality of image data PD1 to P1.
Based on D3 and the initial tag data, the intermediate representation data when the three-dimensional space is measured by one or more other three-dimensional measuring methods is calculated. Then, the tag indicating the result of this operation is described corresponding to a plurality of image data and stored in the database as one or more other tag data. That is, in this example, data for various image-like three-dimensional measurement methods is generated from a large number of omnidirectional image groups and their tag descriptions (initial tag data), and the processing results of the measurement methods are re-described as tag descriptions. Is added to the tag description so far. By this, data of other measurement methods can also be described by this tag description method, and they can be merged to create a large-scale spatial description.

FIG. 5 is a block diagram schematically showing an example of the configuration of the three-dimensional spatial measurement data storage device of the present invention which realizes the method of the present invention. FIG. 6 is a block diagram of this device. It is a flow chart which shows an algorithm of software used in a case. In FIG. 5, 10
Reference numeral 1 denotes a three-dimensional space measuring means including the above-described omnidirectional image measuring device, and the initial image data accumulating means 103 measures the three-dimensional space to be measured by the three-dimensional space measuring means 101 while changing the viewpoint. The plurality of image data obtained as a result are accumulated as initial image data. Then, the landmark check unit 105 automatically checks the correspondence relationship between the plurality of landmarks LM included in each of the plurality of image data. A tag (initial tag) indicating the check result by the landmark check unit 105 is described in XML language corresponding to a plurality of image data and stored in the initial tag data storage unit 107 as initial tag data.

Initial tag data stored in the initial tag data storage means 107 and initial image data storage means 1
If the image data stored in 03 is used, the three-dimensional image of the desired area is displayed by the display means 10 using these data.
8 can be displayed. However, these data alone cannot display an image or process data when a three-dimensional space is measured by another three-dimensional measurement method. For example, the image data of the omnidirectional image and the initial tag data described above cannot display the three-dimensional space in another three-dimensional space display format such as Java3D (registered trademark), and other three-dimensional space. Image software cannot be used either. Therefore, in the present invention, in order to be able to use software that requires data measured by, for example, a measuring method by moving stereoscopic vision, the intermediate representation data calculation unit 109 stores a plurality of data stored in the initial image data storage unit 103. Based on the image data and the initial tag data stored in the initial tag data storage means 107, another n
Intermediate representation data when a three-dimensional space is measured by a three-dimensional measuring method of a type (n is a positive integer of 1 or more) is calculated.

For example, the intermediate representation data calculation means 109
Uses the image data of the omnidirectional image and the above-described initial tag data to previously obtain data measured by a measuring method such as moving stereoscopic vision by calculation (data conversion). For example, the moving stereoscopic vision is a method of measuring the three-dimensional position of the measurement target by using the azimuths of the measurement target from a plurality of points and the position coordinates of the measurement point. The software required for this calculation or conversion can be prepared in advance if the measurement method of another three-dimensional measurement method is known. Therefore, the intermediate representation data calculation means 109 has built-in conversion software necessary for obtaining the measurement result of the three-dimensional measuring method that is frequently used among other known three-dimensional measuring methods. The tag indicating the operation result of the intermediate expression data operation unit 109 is described in the XML language corresponding to a plurality of image data, and is stored in the post-processing tag data storage unit 111 as n types of tag data. Since the intermediate representation data when the three-dimensional space is measured by another three-dimensional measurement method is stored in the post-processing tag data storage means 111 as tag data, these tag data are stored as necessary. It can be processed by the processing means 113.

In the algorithm shown in FIG. 6, the intermediate representation data calculating means 109 is adapted to perform calculation when the three-dimensional space is measured by three other three-dimensional measuring methods. In FIG. 6, steps ST11 and ST12 configure the initial image data storage means 103 in FIG. 5, step ST13 configures the landmark check means 105, and steps ST14 and ST15 configure the initial tag / data storage means 107. The steps ST15 to ST24 constitute the intermediate representation data calculation means 109 and the post-processing tag / data storage means 111.

An example of the calculation method performed by the intermediate expression data calculation means 109 will be briefly described. First, the correspondence of image data and the measurement result are used as the initial tag / data storage means 10.
7, a known landmark LM, an additional landmark LM, and an uncorrelated partial image connecting them are integrated to create an initial environment map. This initial environment map can create a display from any viewpoint.
Also, since the original omnidirectional image is included with the tag, it is possible to create an image group in which the same target is viewed from any measurement point.
In this way, input images for other image-like three-dimensional measurement methods are created, and the results of precise three-dimensional measurement by that method are used as post-processing tags as image description tags (additional tag data). The data is stored in the data storage means 111. At that time, if additional tag data is added in the data description definition in the XML language, the XML language can flexibly respond to the result of the new processing method by adding the definition of the description as needed. As a result, convenience is improved.

Further, as an example of the processing using the database using the initial tag data and the image data, it is possible to use a support tool for the human to manually input the correspondence relationship between the image groups. Measurement result of 3D space by other 3D measurement method using accumulated XML language (spatial description)
Can be displayed by three-dimensional display software such as Java3D (registered trademark) or Web3D. Based on the display, a human can confirm the measurement result, manually correct the measurement result, and add a tag in the three-dimensional space. You can do it without being aware of it.

In the above embodiment, the XML language is used as the extensible markup language used to describe the tag, and its specifications are also defined as described above. However, currently, display description languages for computer graphics are being developed in various countries around the world (for example, Web3
D). The display description language used there can naturally be used in the description of the measurement result data in the present invention. Therefore, in the present invention, the extensible markup language used to describe tags is the above-mentioned X
It is not limited to the ML language. However, as in the above embodiment, from the tab description using the XML language,
Extract images from subsets or multiple viewpoints as needed and process them for other measurement related data processing (eg contour display,
Currently, the XML language is an extensible markup language because it can be easily used for collision detection, action planning, object recognition), display-related data processing (for example, initial creation of indoor data and outdoor data in virtual reality). Is most useful as

As in the above-described embodiment, when a space description is performed using a flexible data description language such as XML language, and conversion into data of various other three-dimensional measurement methods is performed, a wide area is converted. It is possible to provide a means capable of integrating various methods up to precise local space measurement.

The XML language allows the user to define a new data description method as required, and can add to the data up to that point, and the program for processing it can use the existing description method. The data can be processed as an easy extension. Therefore, by using the data description based on such XML language, it is possible to integrate a wide variety of three-dimensional spatial descriptions based on various image processing methods, and various usages according to the purpose. Is possible. Furthermore, since a new image measuring method can also be easily incorporated into this data processing, there is an advantage that the function can be expanded expansively as an open data description.

For reference, a sample space description in XML language is shown in FIG. 7, and an example of XML language definition is shown in FIG.

[0037]

According to the present invention, the intermediate representation data of the three-dimensional space measured by another three-dimensional measuring method is calculated based on the measured image data of the three-dimensional space and the initial tag data, and the calculation is performed. Since the results are described and stored as separate tag data, there is an advantage that even users who do not have knowledge of the 3D measuring device can use various target software using tag data very easily. .

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining the concept of an embodiment when executing a three-dimensional spatial measurement data storage method of the present invention.

FIG. 2 is a flowchart showing an algorithm of a method for determining initial tag data.

3A and 3B are diagrams showing an example of an omnidirectional image in which the positions of landmarks are indicated by black lines extending radially.

4A to 4C are diagrams in which an omnidirectional image is expanded into a cylinder.

FIG. 5 is a block diagram schematically showing an example of the configuration of a three-dimensional spatial measurement data storage device of the present invention that realizes the method of the present invention.

FIG. 6 is a flowchart showing an example of a software algorithm used when implementing this apparatus using a computer.

FIG. 7 is a diagram showing a sample of spatial description in XML language.

FIG. 8 is a diagram showing an example of an XML language definition.

[Explanation of symbols]

1 Omnidirectional image measuring device 3 mobile robots 5 Omnidirectional mirror 101 three-dimensional space measuring means 103 initial image data storage means 105 Landmark Check Means 107 initial tag / data storage means 109 intermediate representation data calculation means 111 Post-processing tag / data storage means

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Yukari Hagiwara             2-24-48-105 Nakamachi, Koganei-shi, Tokyo (72) Inventor Daisuke Mizuno             7-3-1 North 23 West, Kita-ku, Sapporo-shi, Hokkaido             Maison de Ville North Article 23 410 (72) Inventor Keisuke Matsumura             Hokkaido, Sapporo City Chuo-ku Minami 13 West 22-chome 1-38             −505 F term (reference) 2F065 AA04 BB05 FF04 QQ23 UU05                 5B050 AA03 BA04 BA09 BA10 DA07                       EA12 EA13 EA26 FA02 FA19                       GA08                 5B057 AA20 CA12 CB13 CD14 CH01                       CH11 DA06 DA16

Claims (6)

[Claims] 1. A step of acquiring a plurality of image data by accumulating in a database by measuring a three-dimensional space to be measured with a three-dimensional space measuring means while changing a viewpoint, and the three-dimensional space measuring means. A step of describing a tag indicating a correspondence relationship of a plurality of landmarks respectively included in the measured plurality of image data corresponding to a part or all of the plurality of image data, and accumulating in the database as initial tag data. And a step of calculating intermediate representation data when the three-dimensional space is measured by another one or more three-dimensional measurement method based on the plurality of image data and the initial tag data, A step of describing the indicated tag corresponding to the plurality of image data and accumulating in the database as one or more other tag data A method for storing measurement data in three-dimensional space, characterized in that the measurement data in the three-dimensional space is stored by 2. The three-dimensional spatial measurement data storage method according to claim 1, wherein an extensible markup language is used for the description of the tag. 3. The three-dimensional spatial measurement data storage method according to claim 2, wherein the one or more other three-dimensional measurement methods are moving stereoscopic measurement methods. 4. The three-dimensional spatial measurement data storage method according to claim 3, wherein the plurality of image data are a plurality of omnidirectional image data measured by an omnidirectional image measuring device. 5. An initial image data storage unit for storing a plurality of image data obtained by measuring the three-dimensional space to be measured by a three-dimensional space measuring unit while changing the viewpoint, and the plurality of image data storage units. A landmark that automatically checks the correspondence between multiple landmarks included in the image data.
A check unit; an initial tag data storage unit that stores a tag indicating a check result by the landmark check unit corresponding to a part or all of the plurality of image data and stores the initial tag data as the initial tag data; An intermediate expression for calculating intermediate expression data when the three-dimensional space is measured by another n-type (n is a positive integer of 1 or more) three-dimensional measurement method based on the image data of the above and the initial tag data. Data calculation means and tags indicating the calculation results of the intermediate representation data calculation means are described corresponding to the plurality of image data, and n types of tags
A three-dimensional spatial measurement data storage device comprising a post-processing tag and data storage means for storing as data. 6. The three-dimensional spatial measurement data storage device according to claim 4, wherein an extensible markup language is used to describe the tag.