KR20140031129A - Virtual face generating method of omni-directional image using user interaction - Google Patents

Abstract

The present invention relates to a virtual surface generating method of an omnidirectional image based on a user interaction and, more specifically, to a method which easily grasps the position of a structure in an omnidirectional image by generating a grid virtual surface by designating two points on the omnidirectional image that a user obtains and enables the user to obtain a value by calculating the length and area of the structure with an accurate numerical value. For the aforementioned, the virtual surface generating method of an omnidirectional image based on a user interaction according to the present invention includes a designating step, a cross line generating step, a grid virtual surface generating step, a coordinate forming step, a coordinate extracting step, an accumulated information obtaining step, and a calculating step. The designating step designates two points on an image by a user input based on the obtained omnidirectional image. The cross line generating step generates a line which is generated by connecting the two points as a cross line of a virtual surface. The grid virtual surface generating step generates a grid virtual wall surface and a virtual bottom surface by using the determined cross line. The coordinate forming step forms stereoscopic coordinates based on the grid virtual surface. The coordinate extracting step extracts the stereoscopic coordinates of a structure in the omnidirectional image. The accumulated information obtaining step obtains accumulated information about the omnidirectional image. The calculating step calculates the length and area of the structure by using the accumulated information and the stereoscopic coordinates. [Reference numerals] (AA) Start; (BB) End; (S310) Collect overlapped front images and position and posture information; (S320) Calculate an image coordinate value of a conjugate point about a target point; (S330) Calculate an image point vector on a camera coordinate system; (S340) Induce the equation of a line which has a ground point on a ground coordinate system; (S350) Determine the three-dimensional coordinate value of the target point

Description

How to create virtual plane of omnidirectional image based on user interaction {VIRTUAL FACE GENERATING METHOD OF OMNI-DIRECTIONAL IMAGE USING USER INTERACTION}

The present invention is a method for generating a virtual plane of an omnidirectional image based on user interaction, and more specifically, by designating two points on an omnidirectional image acquired by a user to generate a virtual wall surface and a virtual floor surface, the structure of the structure within the omnidirectional image is generated. The present invention relates to a method for easily identifying a location and thus allowing a user to calculate the exact length and area of a structure to obtain a value.

Modern people use a lot of road views to see the actual street through a web browser. Roadview service is a map service that allows users to see the lively streets in three dimensions by taking pictures of real streets around the country with high-resolution panoramas of DSLR cameras.

In general, Street View and Road View of the following are widely known, and are photographed using a specially made photographing equipment and a GPS tracking device capable of capturing 360-degree panoramas. And the stitched images are stitched into a single image. This is the first way before the user travels through a web browser, so it shows the details of what the searcher is looking for when they don't know the road or when they want to see it, and clearly shows restaurant signs, bus stops and even traffic signs. This makes it feel like you are actually going to the street.

However, the road view shows only the appearance of the road, so the information such as the location and distance of each structure is difficult to know. Therefore, in order to know the information such as the location or distance of the structures accurately, the 3D coordinates are used by using an image. The process of obtaining it is necessary.

Accordingly, Korean Laid-Open Publication No. 10-2011-0119589 "System and method for determining three-dimensional coordinates of an object using an omnidirectional image", as shown in Figure 1 (a) the image information acquisition means is a superimposed omnidirectional image Obtaining; (b) acquiring position information and attitude information of the image information acquiring means by the position information acquiring means; (c) calculating, by the user terminal, an image point vector on the camera coordinate system based on the superimposed omnidirectional image, the positional information, the attitude information, and the image point coordinate value of the conjugate point for the predetermined object point; (d) deriving an equation of a straight line in which a ground point exists on the ground coordinate system using the image point vector on the camera coordinate system calculated by the user terminal; And (e) determining a three-dimensional coordinate value for the target point using the intersection calculated by calculating the intersection point by the equation of the straight line derived from the user terminal.

At this time, the three-dimensional coordinates of the object are determined using the overlapping omnidirectional image that can determine the three-dimensional coordinates of the target point by calculating the intersection point of each straight line derived from the conjugated points selected from the two overlapping omnidirectional images.

However, in determining the three-dimensional coordinates in the prior art, since it is calculated based on the orthogonal point of the straight line derived from the conjugate point with two overlapping omnidirectional images, it may not be easy to accurately obtain the three-dimensional coordinates for the single image.

This requires a technique that can accurately determine the position of the structure within the image in the single image as well as the superimposed image.

Korean Publication No. 10-2011-0119589

An object of the present invention is to provide a method for generating a virtual plane of an omnidirectional image based on user interaction, which enables to obtain stereoscopic coordinates corresponding to the position of a structure in the omnidirectional image.

In addition, another object of the present invention is to create a grid virtual surface in which the three-dimensional spatial coordinates are implemented on a user-specified two points on the omnidirectional image, so that the actual length and area of the structure within the omnidirectional image can be easily obtained. It is to provide a virtual surface generation method based on the omnidirectional image.

In addition, another object of the present invention, by using a user-designated two points to be able to freely set the virtual plane that is the reference of the grid coordinates, the virtualization of the omnidirectional image based on the user interaction to increase the degree of freedom of the user It is to provide a method for generating faces.

In addition, another object of the present invention is not to determine the position of the structure in the image having two or more overlapping images, but the user can easily specify the position of the structure in the image by specifying two points on the image with only a single image. It is to provide a virtual plane generation method of the omnidirectional image based on the user interaction that can be identified.

In addition, another object of the present invention, the user can obtain the coordinates by clicking on the image on the content output on the virtual surface, and link the facility information to the acquired coordinates to service the new spatial information It is to provide a virtual surface generation method of the omnidirectional image based on the interaction.

According to a feature of the present invention for achieving the above object, the present invention, the designation step of designating two points on the image by the user input based on the obtained omnidirectional image; An intersection line generation step of determining a line formed by connecting the two points as an intersection line of an imaginary plane; And a grid virtual surface generation step of generating a grid virtual wall surface and a virtual floor surface by using the determined intersection line.

In addition, the present invention is a coordinate implementation step of implementing a three-dimensional space coordinates based on the grid virtual wall surface and the virtual floor surface; And extracting the three-dimensional spatial coordinates of the structure in the omnidirectional image.

In addition, the present invention, the scale information obtaining step of obtaining the scale information for the omnidirectional image; And calculating a length and an area of the structure using the scale information and the three-dimensional space coordinates.

In addition, in the present invention, the step of implementing the content in the omnidirectional image after the calculating step may be further performed.

In addition, the present invention may be characterized in that the area can be calculated by selecting a specific point for the bottom surface of the omnidirectional image when the grid virtual surface generation step.

In addition, the present invention may be characterized in that the image and the video can be inserted according to the size after determining the size of the advertisement for the wall surface of the omnidirectional image when performing the grid virtual surface generation step.

In addition, the present invention may be characterized in that the three-dimensional object can be generated on the bottom surface of the omnidirectional image when the grid virtual surface generation step.

According to the present invention, a method for generating a virtual plane of an omnidirectional image based on user interaction generates a grid virtual plane in which three-dimensional coordinates are implemented by a user designating two points on the acquired omnidirectional image, thereby creating a grid virtual plane in the omnidirectional image. There is an effect that can easily obtain the three-dimensional space coordinates corresponding to the position.

In addition, the present invention, by obtaining the three-dimensional space coordinates, there is an effect that can easily obtain the actual length and area of the structure in the omnidirectional image.

In addition, the present invention has the effect that it is possible to easily obtain the actual length and area of the structure in the image only a single image.

In addition, the present invention has the effect of obtaining the correct length and area, regardless of whether the omnidirectional image is the image of the structure viewed from which angle.

In addition, the present invention can freely set the virtual wall surface and the virtual floor surface serving as the reference for the grid coordinates using two points designated by the user, thereby increasing the degree of freedom of the user.

In addition, the present invention, the user can obtain the coordinates by clicking on the image on the content output on the virtual surface, and can link the facility information to the acquired coordinates to serve new spatial information.

1 is an exemplary view showing a method for determining three-dimensional coordinates in the prior art.
2 is a diagram illustrating an overall flowchart of a method for generating a virtual plane of an omnidirectional image based on user interaction of the present invention.
3 is a diagram illustrating an example in which a user designates two points on an omnidirectional image acquired according to an embodiment of the present invention.
4 is a diagram illustrating an example in which a virtual surface is generated according to an embodiment of the present invention.
5 is a diagram illustrating an example of extracting three-dimensional spatial coordinates of a structure in an omnidirectional image according to an embodiment of the present invention.
6 is a diagram illustrating an example of obtaining and calculating scale information according to an embodiment of the present invention.
7 is a photograph showing a state in which distance measurement is implemented in a method for generating a virtual plane of an omnidirectional image based on user interaction according to the present invention.
8 is a reference diagram of a sphere-plane transformation model for obtaining coordinates for a specific point by a virtual plane generation method of an omnidirectional image based on user interaction according to the present invention.
9 is a photograph showing a state in which the floor surface calculation is implemented in the virtual surface generation method of the omnidirectional image based on the user interaction according to the present invention.
10 is a photograph showing a state in which virtual wall utilization is implemented in a virtual plane generation method of an omnidirectional image based on user interaction according to the present invention.
11 is a photograph showing a state in which an object is generated and a GUI (Graphic User Interface) is implemented in the method for generating a virtual plane of the omnidirectional image based on user interaction according to the present invention.

Hereinafter, a method for generating a virtual plane of an omnidirectional image based on user interaction according to an embodiment of the present invention will be described in more detail with reference to the flowchart shown in FIG. 2.

2 is a virtual plane generation method of an omnidirectional image based on user interaction according to the present invention, a designation step (S100), an intersection line generation step (S110), a grid virtual plane generation step (S120), a coordinate implementation step (S130), The coordinate extraction step S140, the scale information acquisition step S150, and the calculation step S160 are performed.

Step S100 specifies two points on the image by user input based on the acquired omnidirectional image.

Referring to FIG. 3 as an example, the portion marked in red is a point at which the user inputs two points. The position specifying the two points may be anywhere on the image.

In step S110, the two points are connected to each other to create an intersection line of the virtual surface connecting the virtual wall surface and the virtual floor surface.

Step S120 generates a grid virtual wall surface and a virtual floor surface by using the intersection line generated by connecting the two points determined in step S110.

4 shows an example of the generated state of the grid virtual surface, and the grid virtual surface is for obtaining the relative coordinates of the three-dimensional space by tracking the position of the structure in the omnidirectional image.

In operation S130, three-dimensional space coordinates are implemented based on the generated grid virtual plane.

As the coordinates of the three-dimensional space are implemented, the three-dimensional space coordinates of the feature points of all the structures existing in the omnidirectional image can be identified.

Step S140 extracts the three-dimensional space coordinates of the feature points of the structure in the omnidirectional image identified in step S130.

Then, when the extracted three-dimensional space coordinates of the structure is selected, the corresponding position is displayed on the grid virtual plane. FIG. 5 is a process for calculating a window area of a shopping mall existing in the acquired image, and shows an example in which four points of each corner of a window are matched with the grid virtual plane. Thus, since the coordinates of the structures in the omnidirectional image can be easily seen from the virtual wall and the virtual floor, accurate information can be obtained therefrom.

Step S150 obtains scale information on the omnidirectional image.

The three-dimensional spatial coordinates formed on the basis of the grid virtual wall surface and the virtual floor surface are formed in consideration of the scale in which the distance in the area is represented by the distance on the map, and thus scale information from the coordinates of the structures in the omnidirectional image obtained in step S140. Can be obtained.

Step S160 calculates the length and area of the structure using the scale information and the three-dimensional space coordinates.

6 illustrates an example of acquiring scale information, wherein the window edges obtained in FIG. 5 have matching points of the grid virtual plane displayed and clicked on the grid virtual plane. You get the scaled information of 5m in height. It is also possible to obtain information that the area is 40m² using four points.

As the grid virtual surface is formed by the user's input, the position of the structure within the image can be easily obtained by showing the three-dimensional spatial coordinate matching point of the structure in the image on the grid virtual surface. Information such as length, area and volume can be calculated with more accurate values.

Although not shown in the drawing, after performing the calculating step S160, the step of implementing content in the omnidirectional image may be further performed. The content implementing step may include building information such as stairs, elevators, and rooms for each floor. This is the step implemented.

7 shows a state in which a distance measurement is implemented in the virtual plane generation method of the omnidirectional image based on the user interaction according to the present invention.

Meanwhile, in order to generate the virtual wall surface and the virtual floor surface in a specific space when the grid virtual surface generation step S120 is performed, an orthogonal line between the wall surface and the floor surface must be defined in the specific space. Click both end points of the orthogonal line to be made (see Fig. 7 (a)) and connect the two points to define the orthogonal line between the virtual wall and the floor. A virtual bottom surface and a wall surface are generated based on orthogonal lines (see FIG. 7 (b)). Distance measurement (see FIG. 7 (c)) can be performed through the generated virtual wall and bottom surfaces, and two-dimensional coordinates in the image can be expressed in three-dimensional coordinates. This enables calculation with more accurate coordinates and coordinates.

9 shows a state in which a floor calculation is implemented in the virtual plane generation method of the omnidirectional image based on user interaction according to the present invention. In the grid virtual surface generating step (S120), an area may be calculated through the virtual wall and bottom surfaces (see FIG. 9). It is possible to measure the exact width of the polygonal shape through a number of points in the generated floor.

FIG. 10 shows a state in which a virtual wall surface utilization is implemented in the virtual plane generation method of the omnidirectional image based on user interaction according to the present invention. 11 is a photograph showing a state in which an object is generated and a GUI (Graphic User Interface) is implemented in the method for generating a virtual plane of the omnidirectional image based on user interaction according to the present invention.

In the virtual wall and floor generation step (S120), the size, height, etc. of the signboard, which is an advertisement in the building, can be known using the virtual wall, and the image and the video can be inserted to use the advertisement as an advertisement. It is possible. (See Fig. 10)

11 illustrates a state in which an object is created and a GUI (Graphic User Interface) is implemented in the method for generating a virtual surface of an omnidirectional image based on user interaction according to the present invention. It is possible to create an object (see FIG. 11 (a)) by clicking a desired point on the virtual floor surface, and to generate a shadow of a light shining on the object (see FIG. 11 (b)). It is possible to adjust the size of the created object (see FIG. 11 (c)), adjust the rotation angle of the object (see FIG. 11 (d)), and move the object based on the virtual floor surface.

FIG. 8 is a reference diagram of a sphere-plane transformation model for obtaining coordinates of a specific point by a virtual plane generation method of an omnidirectional image based on user interaction according to the present invention.

On the other hand, the three-dimensional space coordinates in step S130 is a coordinate for a specific point on the spherical image obtained by using the spherical-plane transformation model, the orthogonal line for generating the virtual floor surface and the wall surface by the spherical-plane transformation model The coordinates for the two points can be obtained by X = Z / sin (h), cos (h), cos (v), and Y = Z / sin (h), cos (h), sin (v). (See Fig. 8)

In this case, the coordinates may be obtained using the H and V values in the image of the point of the camera height Z and the orthogonal line.

Embodiments have been disclosed through the drawings and the specification. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not used to limit the meaning or the scope of the present invention described in the claims. Therefore, those skilled in the art will implement that various modifications and equivalent other embodiments are possible from this. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (7)

A designation step of designating two points on the image by a user input based on the obtained omnidirectional image;
An intersection line generation step of determining a line formed by connecting the two points as an intersection line of an imaginary plane; And
And a grid virtual surface generating step of generating a grid virtual wall surface and a virtual floor surface by using the determined intersection line.
The method of claim 1,
A coordinate implementation step of implementing three-dimensional space coordinates based on the grid virtual wall surface and the virtual floor surface; And
And a coordinate extracting step of extracting the three-dimensional spatial coordinates of the structure in the omnidirectional image.
The method according to claim 2, wherein
A scale information obtaining step of obtaining scale information of the omnidirectional image; And
And calculating a length and an area of the structure using the scale information and the three-dimensional space coordinates.
The method of claim 3,
And a method of generating content in the omnidirectional image after the calculating step is further performed.
The method of claim 1,
And a method of generating a virtual plane of the omnidirectional image based on user interaction in which an area can be calculated by selecting a specific point with respect to the bottom surface of the omnidirectional image when the grid virtual plane generation step is performed.
The method of claim 1,
And a method of generating a virtual plane of the omnidirectional image based on a user interaction capable of inserting an image and a video according to a corresponding size after determining a size of an advertisement for the wall surface of the omnidirectional image when the grid virtual plane generation step is performed.
The method of claim 1,
And a method of generating a virtual plane of the omnidirectional image based on a user interaction capable of generating a 3D object on the bottom surface of the omnidirectional image when the grid virtual plane generation step is performed.

Images