TWI780734B - Method for displaying hidden object through augmented reality - Google Patents

Abstract

A method for displaying hidden object through augmented reality (AR) includes steps of positioning, capturing plane graphic data, three-dimensional transforming, and synthesizing and displaying. With such steps, position coordinates of a predetermined region are acquired by use of a portable device. The portable device captures the plane graphic data corresponding to the hidden object in the predetermined region from the object database according to the position coordinates thereof. The portable device, transforms the acquired plane graphic data into a three-dimensional graphic data. Then, the portable device captures a real-time image at the site of the position coordinates, so as to synthesize and display the hidden object in the predetermined region according to the three-dimensional graphic data through an AR technique. Therefore, the user is , allowed to clearly know the exact position of the hidden object.

Description

Method for displaying buried objects using augmented reality

The present invention relates to a method for visualizing buried objects, especially a method for displaying buried objects using augmented reality.

With the concept of beautification of the city, many water pipes and electrical lines are mostly buried under the road surface; and for buildings, most of the water pipes and electrical lines are also buried in the wall or floor, so they can not be exposed. so as not to affect the appearance.

However, if there is a need for construction and it is necessary to destroy the road surface or the wall and floor of the building, the construction personnel usually use the planar pipeline distribution map, and then use measurement and visual inspection to estimate the location of the buried pipeline before proceeding. Construction, but this method is judged by the construction personnel based on experience. In addition to being time-consuming, misjudgment of the location often occurs in practice, and there is still a risk of accidentally damaging the pipeline during construction. Therefore, how to solve this problem is The main focus of the present invention lies.

In order to solve the above problems, the present invention provides a method for displaying buried objects using augmented reality, which can display buried objects in live images of mobile devices through augmented reality.

An embodiment of the present invention provides a method for displaying buried objects using augmented reality, which includes the steps of locating, capturing planar image data, stereoscopic transformation, and synthesizing and displaying buried objects. In the step of locating, in a predetermined area, the positioning coordinates of the predetermined area are obtained through a mobile device; in the step of retrieving plane map data, the mobile device obtains from an object database according to the positioning coordinates of the location Retrieving a plane map data corresponding to a buried object in a predetermined area; in the step of stereo conversion, the mobile device stereoscopically converts the buried object according to the plane map data, so as to generate a stereo map data to three-dimensionalize the object; In the step of burying the object, the mobile device captures a live image at the location coordinates, and according to the three-dimensional converted stereoscopic data, the buried object in the predetermined area is synthesized and displayed in the live image by augmented reality.

In this way, the method for displaying buried objects using augmented reality of the present invention uses the location coordinates of the predetermined area as a base through the application of the mobile device, and then captures the planar image data of the buried object and converts it into a three-dimensional image data , that is, in the live image captured by the mobile device phenomenon, the three-dimensional image data is synthesized and displayed through augmented reality, so that through the three-dimensional virtual image, the user of the mobile device can assist the user of the mobile device to visualize the buried objects in the predetermined area It is presented in the live image to facilitate the user's exploration of buried objects and subsequent construction planning.

In order to illustrate the central idea of the present invention expressed in the column of the above-mentioned summary of the invention, it is expressed in specific embodiments. Various objects in the embodiments are drawn according to proportions, sizes, deformations or displacements suitable for illustration, rather than drawn according to the proportions of actual components, which are described first.

Please refer to FIG. 1 to FIG. 7, the present invention provides a method 100 for displaying buried objects using augmented reality, as shown in FIG. The steps of burying the object 104 in this embodiment include a step of image correction 105, which may further include a step of intelligent position comparison 105a, an intelligent image comparison 105b, and an automatic correction 105c. In addition, the mobile device 200 used in this embodiment communicates with the object database 300, the Global Positioning System 400 (Global Positioning System, GPS) and the Geographic Information System 500 (Geographic Information System, GIS), wherein:

In the step of positioning 101, in a predetermined area A, a positioning coordinate C (as shown in FIG. 3 ) where the predetermined area A is located is obtained through a mobile device 200 . The predetermined area A refers to the outdoors in this embodiment, especially on a road section of a street. The buried objects mentioned here are buried pipelines under the road surface, and the common ones are electricity/telecom/network Pipelines, or things like water, sewage, or natural gas lines.

In this embodiment, the mobile device 200 is a tablet computer, but it is not limited thereto, and the mobile device 200 can also be a portable device such as a smart phone or a notebook computer. In this embodiment, the mobile device 200 can enable its own positioning function, and obtain the positioning coordinate C through the Global Positioning System 400 (Global Positioning System, GPS). The mobile device 200 moves and changes at any time, thereby achieving dynamic positioning.

In the step of extracting the plan view information 102 , the mobile device 200 retrieves plan view information of a buried object corresponding to the predetermined area A from an object database 300 according to the location coordinate C. As shown in FIG. 2 , in this embodiment, the mobile device 200 communicates with the object database 300 to retrieve the plane map information. The mobile device 200 obtains the positioning coordinate C through the positioning of the global satellite positioning system 400 , and then the mobile device 200 Through the geographic information system 500, the road information of the predetermined area A where the location is located is identified through the location of the location coordinate C.

Continuing from the above, the object database 300 in this embodiment is a "public facility pipeline database" that has been established by the government agency. Connect the Internet to the object database 300, and you can retrieve the floor plan information of the buried pipeline corresponding to the predetermined area A (as shown in FIG. 3 ). , The distribution of buried pipelines for roads such as running water, sewage or natural gas, and the position of the manhole cover M can also be seen in the floor plan.

In the step of the three-dimensional transformation 103, the mobile device 200 converts the embedded pipeline contained therein according to the planar image data captured from the object database 300 to generate a three-dimensional image data. In the step of synthesizing and displaying the buried object 104, the mobile device 200 captures a real-time image L at the location coordinate C on the spot, and adds the generated stereogram to the live image L to synthesize and display the augmented reality The buried pipeline T in the predetermined area A (as shown in Figure 4).

In this embodiment, the mobile device 200 installs an application program (APP), whereby the application program performs the step of three-dimensional transformation 103 on the captured planar image data, so as to convert the planar image data into three-dimensional image data, and the mobile device 200 Turn on the camera lens through the application program to capture the live image L on the spot. At this time, what you see on the mobile device 200 is the image of the road and the nearby buildings, and then through the steps of synthesizing and displaying the buried object 104, the three-dimensional converted three-dimensional image Synthesized and displayed in the live image L, the three-dimensional buried pipeline T can be virtually displayed from the live image L displayed by the mobile device 200 synchronously.

This embodiment further includes a step of image correction 105, which is to further define specific objects on the road as feature points in the road data, when the live image L is synthesized with augmented reality and displays the buried pipeline and the predetermined area A When the actual position deviates, in the step of image correction 105, through manual input or automatic correction, based on the actual position of the feature point, reposition to correct the buried pipeline T displayed in the live image L and the actual position deviation. The specific object can be a manhole cover, a boundary nail or a utility pole.

Based on the above, since the positioning coordinate C of the person carrying the mobile device 200 in the predetermined area A is not necessarily accurate, if only the positioning coordinate C is used as a reference, the manhole cover M1 and M1 captured in the live image L may be The position of the manhole cover M2 shown in the live image L will be slightly deviated and inaccurate (as shown in FIG. 4 ). At this time, the manhole cover M1 is used as the aforementioned specific object, and the manhole is obtained from the object database 300 The high-precision geographic coordinates obtained after the cover M1 is measured can be used as the feature points, and then corrected so that the positions of the manhole covers M1 and M2 in the live image L overlap without deviation (as shown in Figure 5 ), so that the buried pipeline T can be accurately displayed in the actual position in the live image L.

In a preferred embodiment, the step of image correction 105 includes a step of intelligent position comparison 105a, an intelligent image comparison 105b, and an automatic correction 105c. The step of intelligent position comparison 105a is to locate the coordinate C center, and extract a plurality of specific objects within an extended range, and intelligently compare the relative distance between each specific object and the positioning coordinate C to generate a distance comparison result; the steps of intelligent image comparison 105b are Identify the types of multiple specific objects with intelligent images to generate a type comparison result; through the step of automatic correction 105c, the deviation is corrected with the far-near comparison result and the type comparison result.

For example, the specific object is a manhole cover. During operation, the mobile device 200 obtains the positioning coordinates C through the global satellite positioning system 400, and opens its own camera lens to capture live images L on the spot. The application program The geographical coordinates of the manhole cover in the vicinity will be captured with the positioning coordinate C as the center, and the manpower in the vicinity will be compared with artificial intelligence (for example, using GeoAi geographic artificial intelligence technology) through the step of intelligent position comparison 105a The relative distance between the hole cover and the positioning coordinate C is used to generate the distance comparison result; at the same time, through the image of the complex manhole cover obtained in the live image L captured on site, through the step of intelligent image comparison 105b, intelligent Image recognition The types of manhole covers, such as the images of different types of manhole covers for water supply, gas pipelines, and telecommunications, are built in an image database (not shown in the figure), and are identified through image recognition through intelligent recognition technology The type of the manhole cover in each geographical location is used to generate a type comparison result; then, through the step of automatic correction 105c, the deviation is corrected with the distance comparison result and the type comparison result, which can be automatically corrected The positions of the manhole covers M1 and M2 in the live image L overlap without deviation, so that the buried pipeline T can be accurately displayed in the actual position in the live image L.

It is not difficult to find the characteristics of the present invention from the above description, in that:

1. The method 100 for displaying buried objects using augmented reality of the present invention mainly carries out positioning by carrying the mobile device 200 to the predetermined area A, and uses the positioning coordinates C where the predetermined area A is located as a base to retrieve the buried object The planar image data is converted into a three-dimensional image data, and the three-dimensional image data can be synthesized through augmented reality in the live image L captured by the mobile device 200, and displayed on the mobile device 200, through the three-dimensional virtual The image assists the user of the mobile device 200 to visualize the buried objects in the predetermined area A and present them in the live image L. The user does not need to go through any digging action to achieve the convenience of exploring the buried objects and subsequent construction planning .

2. The present invention further includes the step of image correction 105, because after the step of positioning 101, the position of the buried object (such as the above-mentioned buried pipeline T) displayed in the live image L may have errors from the actual position and is not accurate enough. In the step of image correction 105, the feature points (such as the aforementioned manhole cover with high-precision geographic coordinates) are used as the reference, and the position of the buried object in the live image L can be relocated from the feature points as in the aforementioned embodiment. In order to correct the buried object in the live image L to be displayed at the actual position, especially in the above embodiment, the artificial intelligence correction method is used to further achieve the effect of automatic and precise correction.

The method 100 for displaying buried objects using augmented reality of the present invention is not limited to the implementation of the above-mentioned embodiment in the predetermined area A outside, but in the second embodiment it is implemented in the predetermined area B indoors. In the step 101, the positioning coordinates (not shown in the figure) are not obtained from the positioning of the Global Positioning System 400 (Global Positioning System, GPS), but a gyroscope 201 and an accelerometer provided by the mobile device 200 itself 202 and an altimeter 203 (as shown in Figure 2), use the gyroscope 201 and the accelerometer 202 to measure and detect the position of the person in the predetermined area B, and confirm the position of the person in the predetermined area B with the altimeter 203 Altitude to obtain positioning coordinates. In addition, the floor plan refers to the buried pipeline distribution diagram contained in the indoor design drawing (as shown in FIG. 6 ), and the object database 300 is an independently constructed database in this embodiment. The buried pipeline distribution diagram included in the design drawing is stored therein for the mobile device 200 to access through connection.

In this embodiment, the application program installed on the mobile device 200 also performs the three-dimensional conversion step 103 on the captured planar image data, so as to convert the planar image data into three-dimensional image data, and the mobile device 200 uses the application program Turn on the camera lens to capture the live image L on the spot. At this time, what you see on the mobile device 200 is the wall and floor of the building interior, and then through the steps of synthesizing and displaying the buried object 104, the three-dimensional converted three-dimensional image data is displayed on the live scene. Synthesized and displayed in the image L, the three-dimensional buried pipeline T can be displayed synchronously from the live image L displayed by the mobile device 200 (as shown in FIG. 7 ), and the buried objects in the predetermined area B can also be visualized and presented in In the live image L, the user does not need to go through any excavation actions to achieve the convenience of exploring buried objects and subsequent construction planning.

In addition, in this embodiment, as in the step of image correction 105 in the previous embodiment, the specific objects on the wall panels and/or floors are further defined as feature points in the design drawing, and the specific objects such as wall panels and The boundary of the floor, the socket, the switch or the faucet, in this embodiment, the boundary b between the wall and the floor and the socket E set on the wall are the feature points, when the live image L is synthesized and displayed in augmented reality When the buried pipeline T in the predetermined area B deviates from the actual position, in the step of image correction 105, as described in the first embodiment, the position of the above-mentioned feature points can be used as a reference to reposition to correct the position of the buried pipeline T. The position in the live image L is used to correct the buried pipeline T in the live image L to be displayed in the actual position, and the correction method of artificial intelligence can also be used to further achieve the effect of automatic and accurate correction.

The above-mentioned embodiments are only used to illustrate the present invention, and are not intended to limit the scope of the present invention. All modifications or changes that do not violate the spirit of the present invention belong to the intended protection category of the present invention.

100: method 101: Positioning 102: Retrieve floor plan information 103: Three-dimensional conversion 104:Composite display buried objects 105: Image Correction 105a: Intelligent location comparison 105b: Intelligent image comparison 105c: Automatic correction 200:Mobile devices 201: Gyroscope 202: accelerometer 203: Altimeter 300: object database 400:Global Positioning System 500: Geographic Information Systems A: Reservation area B: Reservation area C: positioning coordinates E: socket L: live video M: manhole cover M1: manhole cover M2: manhole cover T: buried pipeline b: border

Fig. 1 is a flow chart of the method of the embodiment of the present invention. FIG. 2 is a block diagram of the connection between the mobile device and the external communication according to the embodiment of the present invention. 3 is a schematic plan view of buried pipelines retrieved from the object database by the mobile device according to the first embodiment of the present invention. FIG. 4 is a schematic diagram of a mobile device synthesizing and displaying a buried pipeline in a predetermined area through augmented reality according to the first embodiment of the present invention. FIG. 5 is a continuation of FIG. 4 , a schematic diagram of the mobile device synthesizing through augmented reality and displaying that the buried pipeline in the predetermined area is shifted to the right and corrected to the actual position. 6 is a schematic plan view of buried pipelines retrieved from the object database by the mobile device according to the second embodiment of the present invention. 7 is a schematic diagram of a mobile device synthesizing and displaying a buried pipeline in a predetermined area through augmented reality according to the second embodiment of the present invention.

100: method

101: Positioning

102: Retrieve floor plan information

103: Three-dimensional conversion

104:Composite display buried objects

105: Image Correction

105a: Intelligent location comparison

105b: Intelligent image comparison

105c: Automatic correction

Claims (8)

A method for displaying buried objects using augmented reality, comprising the following steps: Positioning: Obtain a positioning coordinate in a predetermined area through a mobile device; Retrieve plan information: the mobile device retrieves plan information of a pair of buried objects in a predetermined area from an object database according to the location coordinates; Three-dimensional transformation: the mobile device three-dimensionally converts the buried object according to the planar image to generate a three-dimensional image; and Synthesizing and displaying buried objects: the mobile device captures a live image at the location coordinates, and synthesizes and displays the buried objects in the predetermined area in the live image according to the stereoscopic data in augmented reality. The method for displaying buried objects using augmented reality as described in Claim 1, wherein the predetermined area is outdoors, and in the positioning step, the mobile device obtains the positioning coordinates through a global satellite positioning system, In addition, the geographic information system identifies the road data of the predetermined area according to the location coordinates; the plane map data is the distribution map of buried pipelines included in the road data. The method for displaying buried objects using augmented reality as described in Claim 2, which further includes a step of image correction, which is to further define specific objects on the road as feature points in the road data, when in the live image When the augmented reality is used to synthesize and display that the buried object in the predetermined area deviates from the actual position, in the step of image correction, the position of the feature point is used as a reference to reposition to correct the deviation. The method for displaying buried objects using augmented reality according to claim 3, wherein the specific object is a manhole cover, a boundary nail or a utility pole. The method for displaying buried objects using augmented reality as described in Claim 3, wherein the image correction step includes an intelligent position comparison, an intelligent image comparison and an automatic correction step, the intelligent position comparison The step is to take the positioning coordinates as the center and extract a plurality of the specific objects within an extended range, and intelligently compare the relative distances between each of the specific objects and the positioning coordinates to generate a distance comparison result; the intelligent image The step of comparing is to identify the types of the plurality of specific objects with intelligent images to generate a type comparison result; the step of automatic correction is to correct the deviation according to the far-short comparison result and the type comparison result. The method for displaying buried objects using augmented reality as described in Claim 1, wherein the predetermined area is indoors, and in the step of positioning, it is detected by a gyroscope and an accelerometer of the mobile device The orientation of the person in the said room, and the positioning coordinates obtained by confirming the height of the person in the said room with an altimeter; the floor plan is the distribution map of buried pipelines contained in the design drawing of the said room. The method for displaying buried objects using augmented reality as described in claim 6, further comprising a step of image correction, which is to further define specific objects on the wall and/or floor as feature points in the design drawing, When the live image is synthesized by augmented reality and shows that the buried object in the predetermined area deviates from the actual position, in the step of image correction, repositioning is based on the position of the feature point to correct the described deviation. The method for displaying buried objects using augmented reality as claimed in claim 7, wherein the specific object is a boundary of a wall or a floor, a socket, a switch or a faucet.

Images