KR20110068764A - System and method for providing object information from aspherical image acquired by omnidirectical camera - Google Patents

Abstract

PURPOSE: An object information supplying system and method thereof which uses a spherical image in all directions are provided to rapidly supply three dimensional information and data information through object information within an image. CONSTITUTION: An image database(11) stores a spherical image from a front camera of a moving unit and a GPS(Global Positioning System)/INS(Inertial Navigation System) apparatus. An object information database(13) stores object information within a photographing area. A display module(20) displays a spherical image of the image database. A matching unit(30) overlaps object information of the object information database in a displayed spherical image.

Description

System and method for providing object information using spherical images acquired from omnidirectional cameras {System and Method for providing object information from aspherical image acquired by omnidirectical camera}

The present invention relates to a system and a method for providing object information of objects photographed on spherical images using spherical images acquired from a omnidirectional camera. More particularly, the present invention relates to a mobile unit equipped with a omnidirectional camera and a GPS / INS device. While acquiring the spherical image and the position and posture information of the camera, after acquiring the object information of the objects existing in the shooting area, the spherical image and the object information are superimposed on the display module, and the object not acquired as the object information. The present invention also relates to a method for automatically extracting and displaying three-dimensional information of the system.

With the rapid development of image information technology, various products that display information by superimposing the information on the image and the objects appearing on the image have been released.

For example, Korean Patent No. 687990 "Image-Based Navigation System and Method" presented a technique for acquiring location information of a mobile terminal and mapping and displaying previously photographed image information on a digital map.

Recently, Patent No. 822814 "Spatial Information Service Method Combining Surveying Information, GIS Geographical Information, and Real-Time Image Information Using GPS / INS Equipment" describes a technique of displaying information on objects in real-time with imaging. Presented.

The former patent is to extract the route from the starting point to the destination from the numerical map, and then display the actual image of the corresponding route photographed in advance according to the position, and simply display the attribute information of the objects on the displayed image.

The latter patent is to obtain the position and attitude (shooting direction) information of the camera by using the GPS / INS equipment, and then extract the information of the objects that can be included in the image therefrom, and the absolute coordinates of the extracted objects from the database The present invention relates to a technique of reading and substituting the absolute coordinates of the read objects into a collinear conditional expression to extract the image coordinates of the objects from the image, and displaying various information of the objects on the extracted image coordinates with the image.

Although the two patents are obvious in terms of realism, realism, convenience, and usefulness in that they overlap information of real images and objects included in the images, the object information that can be displayed by being superimposed on the images is stored in a database. It is limited in that it is only information that is already stored.

The database that stores the object information of the objects, including the object information of the numerically mapped objects, includes the names of the objects (e.g. street lights, street trees, buildings, manholes, fire hydrants, etc.) and the absolute coordinates, addresses, and mutual names of the representative points of the objects. Only new construction time and renovation time are stored, and the specific three-dimensional information of the objects is not stored. And the database of specific three-dimensional information of objects itself is almost impossible. In other words, when an object is a building, it takes tremendous time and money to database all three-dimensional information such as the absolute coordinates, area, and volume of the building edges, and the absolute coordinates for any particular point that is not an edge can be databased. Also, it is impossible to extract the absolute coordinates of a specific point even by using the collinear condition equation widely used in the field of imaging technology.

As described above, in the conventional art of displaying an image photographed by overlapping object information of objects included in the image, only the object information previously stored in the database is displayed by overlapping the image and not providing more detailed 3D information. The utilization was low.

The present invention further uses the spherical image obtained from the omnidirectional camera, which can automatically extract and show specific three-dimensional information of the objects included in the image, in addition to simply superimposing only the object information stored in the database as described above. The object of the present invention is to provide an individual information providing system and a method thereof.

The object information providing system using the spherical image obtained from the omnidirectional camera according to the present invention for achieving the above object

An image DB storing spherical images obtained from the omnidirectional camera and the GPS / INS equipment mounted on the vehicle and the position information and the attitude information of the omnidirectional camera;

An object information database in which object information of objects existing in the area photographed by the omnidirectional camera is stored;

A display module for displaying a spherical image stored in the image DB;

Matching means for superimposing the information of the objects stored in the object information DB on the spherical image displayed on the display module;

An input unit for receiving a command signal of a user;

A control unit controlling spherical images and object information displayed on the display module according to a command signal input from the input unit;

When a specific object displayed on the display module is selected through the input unit, a specific point of the specific object selected by using the [sphere-plane conversion model] and the image change characteristic of the spherical image (that is, the difference in color and contrast of the image) Characterized in that it further comprises; three-dimensional information extraction unit for extracting the three-dimensional ground coordinates (X, Y, Z) for.

And the object information providing method using the spherical image obtained from the omnidirectional camera according to the present invention

 (S10) obtaining surrounding images from the omnidirectional camera mounted on the moving means;

(S20) acquiring position information and attitude information of the omnidirectional camera from the GPS / INS equipment mounted on the moving means;

(S30) acquiring a spherical image from the surrounding images obtained from the omnidirectional camera by using the position information and the attitude information acquired from the GPS / INS equipment;

(S40) acquiring object information of objects existing in the area photographed by the omnidirectional camera;

(S50) superposing the spherical image acquired in the step (S30) and displaying the object information obtained in the step (S40) on the display module;

(S60) if a specific object is selected from the spherical image displayed on the display module, extracting and displaying the 3D information of the selected specific object and superimposed on the spherical image of the display module;

In the case of selecting a building as a specific object in step S60, before extracting 3D information of the selected specific object.

(S60a) if one side of the building is preselected in the spherical image, superimposing the outline on the spherical image with the outline of the representative name of the building and the edge of one side of the preselected building as object information of the building;

(S60b) After the step (S60a), if one surface of the preselected building is selected and selected from the spherical image, the spherical image may further include moving to an image of one surface of the selected corresponding building as the highest surface. and,

The step (S60) is

(S61) The Z0 value at the known coordinates (X0, Y0, Z0) in the spherical image, and the φ and λ values, which are the pan and tilt angles in the spherical image with respect to a specific point of the selected specific object, are determined by the spherical-plane transformation model. ] To find the coordinate X and Y values for a specific point,

(S63) Under the assumption that the change of the feature from the known point to the specific point in the spherical image is changed only horizontally and vertically,

By using the image change characteristics (that is, the difference between the color and the contrast of the image) from the known point to the feature, the features are distinguished from each other in the image, and the boundaries of the horizontal and vertical parts in the same feature,

And a step of obtaining an altitude value Z for a specific point of the spherical image from the sum of the lengths of the vertical lines after connecting the separated horizontal feature and the boundary from the known point of the image to the specific horizontal horizontal line and the vertical line. It features.

The object information providing system and the method using the spherical image obtained from the omnidirectional camera according to the present invention having such a configuration provides an image of the 360-degree direction in one place by using the spherical image, the object included in the image It is an invention useful for industrial development that can be usefully and conveniently used in various fields because it automatically extracts and provides not only information stored in the database as object information but also 3D information not stored in the database.

1 is a block diagram of an object information providing system using a spherical image obtained from the omnidirectional camera according to the present invention.
2 is a procedure of the object information providing method using the spherical image obtained from the omnidirectional camera according to the present invention.
3 is an example of a spherical image.
4 is a perspective view of the omnidirectional camera according to an example.
5 is an example of spreading a sphere in a plane to explain the [sphere-plane conversion model].
6 is a view for explaining a method of extracting the ground coordinate Z;
7 is an example of spherical image and object information provided by the present invention.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

As shown in the block diagram of FIG. 1, the system according to the present invention includes a database 10, a display module 20, a matching means 30, an input unit 60, a control unit 50, and a three-dimensional information extraction unit 40. )

The database 10 has a spherical image taken while the omnidirectional camera mounted on the moving means is moved, and an image in which positional information and attitude information of the omnidirectional camera acquired by the GPS / INS device mounted together with the omnidirectional camera are stored every time. DB 11,

An object information DB 13 for storing object information about objects existing in an area photographed by the camera,

The other DB 15 which stores various programs or data necessary for the operation of the present system is included.

The display module 20 displays the spherical image stored in the image DB 11 under the control of the controller 50 receiving the command signal from the input unit 60, and displays the spherical image in the object information DB 13. The object information of the objects included in the displayed spherical image among the stored object information is superimposed on the spherical image and displayed, and the 3D information extractor 40 displays the 3D information of the objects extracted on the spherical image.

The matching means 30 matches the information of the objects stored in the object information DB with the corresponding object of the spherical image and displays the superimposed image on the spherical image. The object information includes attribute information such as a name, an address of an object, an identification number for identifying the same object, and ground coordinates for indicating the location of the object.

In particular, the ground coordinates of an object are the absolute coordinates of a specific representative point of the object, which include latitude, longitude, and altitude, but in some cases, absolute coordinates without latitude values (ie latitude and longitude) are provided. do. The representative point of the object is usually one of the points of contact with the ground. In addition to the representative point of the object, the ground coordinates of the corner points of the object may be included.

The object displayed in the spherical image is matched with the object stored in the object information DB 13 using the absolute coordinates of the object representative point. The matching method uses a collinear condition equation representing the relationship between the image coordinates (i, j) of the image and the ground coordinates (x, y, z) of the actual ground. More specifically, when the absolute coordinates (x, y, z) of the representative points of the objects stored in the object information DB are substituted into the collinear condition equation, it is calculated which image coordinates (i, j) are in the spherical image. When the image coordinates of the object are calculated as described above, the various coordinate information of the object is superimposed and displayed on the calculated image coordinate of the spherical image. And when the ground coordinates provided by the object information database are given only latitude and longitude and no value for altitude, the position information acquired by the GPS / INS mounted on the vehicle, that is, the altitude value of the road surface through which the vehicle passes. Calculate the image coordinates of the object by substituting into the collinear condition equation as the altitude value of the object. This is possible because most of the known points of the objects are in contact with the ground, and the altitudes of the known points of the object and the known points of the vehicle (ie, the road surface) do not differ significantly.

For reference, the collinear condition equation has two equations, and the variables are the image coordinates (i, j) and the world absolute coordinates (x, y, z). Therefore, if you know the absolute coordinates, the image coordinates are calculated, but if you know the image coordinates, the absolute coordinates are not calculated. Therefore, when calculating absolute coordinates from image coordinates using collinear conditions, two images with the same object must be present.

That is, in the general method, it is not possible to extract the actual ground coordinates of a specific point of the image from one image, and it is the three-dimensional information extraction unit 40 of the present invention that enables this. A detailed description of the three-dimensional information extraction unit will be described later in the object information providing method according to the present invention. Here, the three-dimensional information extraction unit simply provides the three-dimensional information under the condition that the object information is sufficiently provided in the object information database. It briefly describes how to extract.

When the object information database provides three-dimensional ground coordinates (x, y, z) for all corner points of an object, the object's width, height, area, volume, etc. can be used using well-known distance formulas or area formulas. Three-dimensional information can be easily obtained. However, even in this case, unless the object is a building, a specific point is the top or bottom of a few floors of the building, and the object information DB provides information about the number of floors of the building and the height of each floor, the object information DB Information about the height of a specific point and not the edge of an object that is not provided is not available in the usual way.

The input unit 60 receives a user's command signal and transmits it to the controller. An input key, a mouse, or the like may be used as the input unit.

The control unit 50 receives a command signal from the input unit to control the spherical image and the object information displayed on the display module 20, the overall control of the system of the present invention.

7 shows an example of a display module screen.

As shown in FIG. 7A, a spherical image photographed by the omnidirectional camera is displayed at the center of the screen, and object information on the area around the area displayed as an image is displayed on the right side in the form of an electronic map, and a command signal of the user is displayed on the left side. The command signal input window is displayed for easy input.

On the electronic map on the right, the blue route on the road shows the movement route of the vehicle, the location of the spherical image displayed by the red arrow, and the trees and street lamps are displayed in green on both sides of the road. The building and its name are marked. Here, the movement path of the moving means and the current position display of the moving means are displayed in association with the data of the image DB 11, and the others are displayed as data of the object information DB.

In the input window on the left, the driving direction selector selects the moving direction of the moving means, and if there is a spherical image photographing the same area in the past, the current spherical image and the past spherical image are selected and displayed. That is, there is a past picture comparison unit that can compare the change of the object), and a bookmark unit that displays a list of the representative buildings around the user so that the selected building is displayed as spherical images when the user selects it.

In the center spherical image, circles in the form of silhouettes are continuously displayed on the road, which indicates the direction of movement of the vehicle equipped with the omnidirectional camera.

 Looking at the side of the road in FIG. 7A to FIG. 7B, a circle of white solid line whose diameter is changed is indicated. This indicates a matched position in the spherical image of the object stored in the object information DB. When the user moves the cursor on the screen to the white circle, the name of the object 'the number of trees' is superimposed on the spherical image.

7C and 7D, a red circle is displayed on the screen, and the spherical image shows the right side if FIG. 7C is displaying the front side. The user rotates the spherical image by dragging the cursor on the screen. Since the omnidirectional camera acquires spherical images by simultaneously photographing all four directions from one location, the image can be rotated in this way.

In FIG. 7E, when the user clicks 'Charan' in the favorite part of the input window, the spherical image moves and displays the area around the building called 'Charan' as shown in FIG. 7F, and at the same time, the red arrow indicates the current position in the electronic map on the right. Is displayed again.

When the user places the cursor in the building as shown in FIG. 7G, the object information 'difference' is displayed on the building, and the exterior line is displayed on the edge of the building surface where the cursor is located, and as shown in FIG. Move back to see the building face where the cursor is located.

Hereinafter, a description will be given of the object information providing method according to the present invention and the role of each component therefor.

In the object information providing method using the spherical image obtained from the omnidirectional camera according to the present invention, as shown in FIG. 2, the method includes obtaining an image from the omnidirectional camera mounted on the moving means (S10), and the position information and the attitude information from the GPS / INS equipment. Acquiring (S20), acquiring a spherical image (S30), acquiring object information on the photographing area (S40), and superimposing the acquired spherical image and object information on the display module (S50). If a specific object is selected in the display module, the method may include extracting and displaying 3D information of the object (S60).

In step S10, the surrounding image is acquired from the omnidirectional camera mounted on the moving means. The omnidirectional camera is a camera for capturing an image of the surrounding omnidirectional (front, rear, left and right directions) at once. In general, as shown in FIG. It is.

In step S20, position information and attitude information of the GPS / INS device are acquired from the GPS / INS device.

The omnidirectional camera and the GPS / INS equipment are mounted together in a moving means such as a vehicle, and continuously acquire the surrounding image, the position information, and the attitude information as the moving means moves. The location information and attitude information acquired by the GPS / INS device are the location information and the attitude information of the GPS / INS device itself. However, since the omnidirectional camera and the GPS / INS device are mounted on the same moving object, when the error is corrected, Posture information or moving object position information and posture information.

In the step S30, the spherical image is acquired from the surrounding images captured by the omnidirectional camera using the position information and the attitude information in the previous step.

Since a method for acquiring spherical images from surrounding images captured by the omnidirectional camera is a well-known technique, a detailed description thereof will be omitted.

A spherical image is generated for each position of the omnidirectional camera, and FIG. 3 is an example of a spherical image acquired from surrounding images photographed from the omnidirectional camera, and is a spherical image with the omnidirectional camera positioned in the center of FIG. 3.

In step S40, object information of objects existing in the area photographed by the camera is acquired. Object information is already widely provided in the form of digital maps and electronic maps.

Provided object information includes property information such as name, address, purpose, phone number, and ground coordinates that inform the location of the building when the object is a building. As described above, the ground coordinates do not provide fine details. It is also practically difficult or impossible to provide things.

The ground coordinates provided as object information are three-dimensional ground coordinates (x = latitude, y = longitude, z = altitude) for the representative point of the object (typically the point where the object meets the ground), and a specific point (eg Ground coordinates for corner points of a building, and ground coordinates for a particular point generally provide only latitude and longitude, not elevation. Furthermore, neither ground nor coordinates can be provided for any point selected by the user, which is not a corner.

As described above, the 3D information extracting unit extracts 3D ground coordinates of an arbitrary point of the object not provided as object information and extracts 3D information of the object using the extracted ground coordinates. This will be described again in step S60 below.

In step S50, as shown in FIG. 7, the spherical image acquired in step S30 and the object information acquired in step S40 are superimposed and displayed on the display module under the control of a controller that receives a user's command signal input through the input unit.

In step S60, when a specific object is selected from the spherical image displayed on the display module, the 3D information extractor extracts 3D information of the selected object and displays the 3D information on the spherical image of the display module.

In step S60, if the selected object is a building, the spherical image showing the best view of the building from the spherical image is displayed on the display module before extracting 3D information on a specific point of the building.

This process is described below with reference to FIGS. 7F to 7H.

When the user preselects one side of a specific building in the spherical image currently displayed as shown in FIG. 7F, the representative name of the building as object information of the building is displayed on one side of the preselected building as shown in FIG. 7G (a difference in the drawing). And an outline is displayed on the edge of one side of the pre-selected building, superimposed on the spherical image (S60a). In this case, as a preliminary selection, as shown in FIG. 7G, the cursor may be positioned on one surface of the building in the spherical image.

After checking the representative name and the outline of the building superimposed on the spherical image, if the user confirms and selects one side of the pre-selected building in the spherical image, the display module moves the currently displayed spherical image and the one side of the selected building as shown in FIG. In order to display the spherical image seen in front (S60b) (S60b). The definite selection here can be used in one way by clicking the cursor on the corresponding side of the building.

The three-dimensional information includes three-dimensional ground coordinates of an arbitrary point of the object selected by the user in the spherical image, height, area, and volume of the selected object. Height, area, volume, etc. must first be extracted three-dimensional ground coordinates for specific points of the object.

The 3D ground coordinates (X, Y, Z) of the arbitrary points or specific points selected in the spherical image are obtained by using the [Sphere-Plane Transformation Model] in step S61. Find the Z value using the change characteristic. This will be described in more detail below.

[Sphere-Plane Transformation Model] shows the relationship when the sphere is unfolded as shown in Fig. 5, which is the radius (N), the pan value (φ) and the tilt value (λ), which are coordinates of the sphere. It is a model for obtaining three-dimensional ground coordinates (X, Y, Z), which are rectangular coordinates.

And the [sphere-plane transformation model] representing these relations, that is, the relationship between the coordinates of the sphere and the rectangular coordinates of the plane, is expressed by the following equation.

X = N, cosφ, cosλ

Y = N, cosφ, sinλ

Z = N

or,

φ = arctan (Z / P)

λ = arctan (Y / Z)

Where P = (X ^ 2 + Y ^ 2) ^ (1/2).

In the spherical image, the radius N of the sphere is unknown. That is, it is not possible to determine whether to make the ground of the three-dimensional ground sphere, the building the sphere, or the sky the sphere in the spherical image photographed in three dimensions. In addition, in the present invention, since the N value is not directly used to extract the ground coordinates, setting the spherical reference has no meaning.

In extracting the ground coordinates from the spherical image, since N is not directly used, the equation of the [spherical-plane conversion model] is modified as follows.

Z = N · sinφ ☞ N = Z / sinφ

X = Z / sinφ · cosφ · cosλ

Y = Z / sin φ cos φ sin λ

As can be seen from the modified equation, the ground coordinates X and Y for a specific point in the spherical image are represented by the pan value φ, tilt value λ, and Z value of the specific point in the spherical image.

The pan value φ and the tilt value λ are values for the coordinates of the selected point (i.e. specific point) in the spherical image, and can be obtained immediately by selecting any point in the image. Therefore, the Z value of the selected point is known or specified. Assuming, we can get X and Y values.

In other words, if you already know the Z value of the selected point, use [Sphere-Plane Transformation Model] to find the X, Y value of the selected point.

If the Z value of the selected point is not known, the Z value of the selected point is assumed and then the X and Y values of the selected point are obtained using the [Spherical-Plane Transformation Model]. At this time, the Z value is assumed to be the Z value of the known point known in the spherical image, and the X and Y values obtained using the [Spherical-Plane Transformation Model] are moved to the vertical direction from the selected point (that is, the known point). X and Y of the point having the same altitude (Z) and are equal to each other. In this step, we want to find the X and Y values of the three-dimensional ground coordinates, and the Z values of the three-dimensional ground coordinates are obtained in the other steps. It can be.

And since the vehicle is equipped with GPS / INS equipment, the altitude value of the camera can be obtained by using the position and posture information acquired by the GPS / INS equipment.If the installation height of the camera is subtracted from the road surface, the Z value of the road surface is obtained. Can be obtained. Since the road surface Z value of this specific point is acquired at the same time as the image acquisition, it is always present and available in the spherical image, so it is based on the road surface of the path traveled by the moving object when obtaining the ground coordinates X and Y values of the selected point. To estimate the Z value.

For reference, the coordinates X, Y, and Z obtained in this step S61 are coordinates based on the spherical image, and are not absolute ground coordinates based on the earth ellipsoid. However, when the rotation and movement transformation is performed using the absolute coordinates and attitude information of the camera obtained from GPS / INS, the coordinates based on the spherical image are easily converted to the absolute ground coordinates based on the ellipsoid of the earth. In addition, since the rotation transformation and movement transformation used herein correspond to well-known and common techniques well known in the technical field to which the present invention belongs, the coordinates X, Y, and Z obtained in this step (S61) and the absolute ground coordinates based on the earth ellipsoid There is no problem even if identified. Thus, in describing the present invention, the coordinates X, Y, and Z (especially X, Y) obtained in step S61 may be represented by ground coordinates.

In this way, after obtaining the ground coordinates X and Y values of a specific point (φ, λ) on the spherical image by using the known Z value of the known point in the spherical image (S61), the characteristics of the ground feature changes on the ground. Obtain the ground coordinate Z of the specific point (S63).

Most of the land we live on is flat, and most of the features on the ground are vertical.

Therefore, after adding the assumption that the ground feature changes only horizontally and vertically, and then connecting the virtual horizontal line and vertical line continuous from the known point of the spherical image to a specific point, the specific point of the spherical image from the sum of the lengths of the vertical lines We can get ground coordinate Z for.

For reference, the distinction between features in the image and the boundary between the horizontal and vertical portions in the same feature can be easily distinguished by the color and contrast of the image.

That is, when the bar and the image shown in FIG. 6 include roads and sidewalks, buildings and power poles on the sidewalks, and the user selects a specific point of the pole,

The camera-mounted vehicle connects the virtual horizontal line and the vertical line to a specific point selected using a point on the road surface passing by the camera. That is, the first horizontal line r1 is connected from the known point to the end of the road surface, the first vertical line r2 is connected to the upper side of the sidewalk, and the second horizontal line r3 is connected from the upper side of the sidewalk to the lower side of the power pole, The second vertical line r4 is connected to the selected specific point.

That is, only the X and Y values of the ground coordinates and the X and Y values of the ground coordinates change on the horizontal line, only the Z values of the Z axis of the ground coordinates change on the vertical line, and the ground coordinates of the known point (assuming (X0, YO, Z0)) are already As you know, the ground coordinates (X1, Y1, Z0) of the point where the first horizontal line (r1) and the first vertical line (r2) meet, the ground coordinates of the point where the first vertical line (r2) and the second horizontal line (r3) meet ( X1, Y1, Z1), the ground coordinates (X2, Y2, Z1) at the point where the second horizontal line r3 and the second vertical line r4 meet, and the ground coordinates (X2, Y2, Z2) at the selected point are sequentially obtained. Can be.

And the known point may be preferable to reduce the error to select the nearest point to the selection point.

In the above, obtaining 3D ground coordinates (X, Y, Z) for an arbitrary point selected by a user has been described. The 3D information of an object (ie, an object), for example, the height, width, area, Most of the building points needed to obtain the volume and the like are the corners, the top and the bottom of the building, and these can be easily selected automatically from the image without being directly selected by the user. Therefore, the points necessary for 3D information extraction do not need to be selected by the user, and the 3D ground coordinates (X, Y, Z) extraction of the points are performed by the 3D ground coordinates (X, Y, Z) Same as extraction process.

Through this process, the process of obtaining three-dimensional ground coordinates X, Y, and Z for an arbitrary point of the selected object (ie, object) on the spherical image is briefly described again.

After the spherical image is acquired, if you click (select) any point of interest in the spherical image, the coordinates of the clicked point (that is, the angle represented by pan and tilt) are extracted directly from the spherical image. The sphere radius N is also extracted directly from the known Z of the replacement point. In the spherical image, the known point becomes the ground of the movement path of the vehicle (moving body) equipped with the GPS / INS equipment. That is, the absolute coordinates (Xa, Ya, Za) of the GPS / INS equipment are acquired by the GPS / INS equipment in real time, and the installation height H of the GPS / INS equipment is already known in the moving object, so that the ground (ie ) Z value is Za-H.

When you click (select) a point in the spherical image, the pan and tilt values of the spherical image and the Z value of the known point that replaces the spherical radius of the spherical image are extracted immediately. ] Is applied to obtain the X and Y values of the relative coordinates of the selected point in the spherical image.

As mentioned above, these X, Y, and Z are relative positions from the camera (i.e., X, Y, and Z are coordinates based on the spherical image, which indicates the direction and distance from the camera). Based on the absolute position of the camera obtained from the INS, it is obtained by the absolute ground coordinates of the selected point through a rotation transformation using the attitude information of the camera.

After obtaining the absolute coordinate value of the clicked point in the spherical image, the coordinate values are obtained by using the image change characteristic of the location of various facilities of the image. As shown in Fig. 6, when the point to acquire the position is any point on the power pole, the known point in the spherical image is on the road which is the ground which is the path through which the moving object passed. In the drawing, the manhole cover on the road can be one of the known points. In the manhole cover of the known point, connect the virtual straight lines to a point on the power pole at the point where the position is to be acquired. The virtual straight lines are assumed to be changed only in the horizontal and vertical directions in the image feature. . This is an assumption that can be recognized because most of the features on the ground are installed vertically.

We can then find the change in the coordinates (X, Y, Z) of the horizontal and vertical lines, a virtual straight line connecting the known point on the manhole to the pick point on the pole. In other words, the Z value does not change on the horizontal line, only the Z value changes on the vertical line, and we finally obtain the position value that we want to obtain.

And, if the height r2 of India is known in Fig. 6, the Z value of the reference point applied to the [sphere-plane transformation model] is used instead of the Z value of the road surface, and r3 and r4 If you select the meeting point with a single click, you can obtain the three-dimensional coordinates of the power pole through the S61 process. Of course, the coordinates obtained at this time are the coordinates of the point where the power pole meets the sidewalk, and when selecting any point on the power pole as shown in FIG. 6, the height (Z value) of the selected point must be acquired through step S63.

When the three-dimensional ground coordinates (X, Y, Z) for the points required in the object are extracted, the three-dimensional information required in step S65 is extracted.

In the above description of the present invention, a system and method for providing object information using a spherical image obtained from a omnidirectional camera having a specific configuration and a specific procedure are described with reference to the accompanying drawings. It is to be understood that such modifications and variations are intended to be included within the protection scope of the present invention.

10: Database 11: Image DB
13: Object Information DB 20: Display Module
30: matching means 40: three-dimensional information extraction unit
50: control unit 60: input unit

Claims (5)

An image DB storing spherical images obtained from the omnidirectional camera and the GPS / INS equipment mounted on the vehicle and the position information and the attitude information of the omnidirectional camera;
An object information database in which object information of objects existing in the area photographed by the omnidirectional camera is stored;
A display module for displaying a spherical image stored in the image DB 11;
Matching means for superimposing and displaying the information of the objects stored in the object information DB to the spherical image displayed on the display module; object information providing system using a spherical image obtained from the omnidirectional camera. The method of claim 1,
An input unit for receiving a command signal of a user;
A control unit controlling spherical images and object information displayed on the display module according to a command signal input from the input unit;
When a specific object displayed on the display module is selected through the input unit, a specific point of the specific object selected by using the [sphere-plane conversion model] and the image change characteristic of the spherical image (that is, the difference in color and contrast of the image) 3D information extraction unit for extracting the three-dimensional ground coordinates (X, Y, Z) for the object information providing system using a spherical image obtained from an omnidirectional camera. (S10) obtaining surrounding images from the omnidirectional camera mounted on the moving means;
(S20) acquiring position information and attitude information of the omnidirectional camera from the GPS / INS equipment mounted on the moving means;
(S30) acquiring a spherical image from the surrounding images obtained from the omnidirectional camera by using the position information and the attitude information acquired from the GPS / INS equipment;
(S40) acquiring object information of objects existing in the area photographed by the omnidirectional camera;
(S50) superposing the spherical image acquired in the step (S30) and displaying the object information obtained in the step (S40) on the display module;
(S60) if a specific object is selected from the spherical image displayed on the display module, extracting the three-dimensional information of the selected specific object and superimposed on the spherical image of the display module; and displays the spherical image obtained from the omnidirectional camera Object information provision method using. The method of claim 3, wherein
In the case of selecting a building as a specific object in step S60, before extracting 3D information of the selected specific object.
(S60a) if one side of the building is preselected in the spherical image, superimposing the outline on the spherical image with the outline of the representative name of the building and the edge of one side of the preselected building as object information of the building;
(S60b) After the step (S60a), if one surface of the preselected building is selected and selected from the spherical image, the spherical image may further include moving to an image of one surface of the selected corresponding building as the highest surface. Object information providing method using the spherical image obtained from the omnidirectional camera. The method of claim 3 or 4, wherein the step (S60)
(S61) The Z0 value at the known coordinates (X0, Y0, Z0) in the spherical image, and the φ and λ values, which are the pan and tilt angles in the spherical image with respect to a specific point of the selected specific object, are determined by the spherical-plane transformation model. ] To find the coordinate X and Y values for a specific point,
(S63) Under the assumption that the change of the feature from the known point to the specific point in the spherical image is changed only horizontally and vertically,
By using the image change characteristics (that is, the difference between the color and the contrast of the image) from the known point to the feature, the features are distinguished from each other in the image, and the boundaries of the horizontal and vertical parts in the same feature,
And a step of obtaining an altitude value Z for a specific point of the spherical image from the sum of the lengths of the vertical lines after connecting the separated horizontal feature and the boundary from the known point of the image to the specific horizontal horizontal line and the vertical line. A method for providing object information using spherical images acquired from an omnidirectional camera.

Images