CN116229028A

CN116229028A - AR-based construction indication method and device, electronic equipment and storage medium

Info

Publication number: CN116229028A
Application number: CN202310232031.9A
Authority: CN
Inventors: 闫楠
Original assignee: Beijing Hetu United Innovation Technology Co ltd
Current assignee: Beijing Hetu United Innovation Technology Co ltd
Priority date: 2023-03-10
Filing date: 2023-03-10
Publication date: 2023-06-06

Abstract

The invention discloses an AR-based construction indication method, an AR-based construction indication system, electronic equipment and a storage medium. The method comprises the following steps: determining UTM position information of the terminal equipment under a UTM coordinate system based on GPS position information of the terminal equipment; determining information of one or more hidden projects existing in a set range around the terminal equipment according to UTM position information of the terminal equipment; acquiring 3D model resources of the one or more hidden projects from a preset resource library according to the information of the one or more hidden projects; determining that a first hidden project exists in the current visual field range of the terminal equipment based on the current azimuth information of the terminal equipment; determining height information of the terminal equipment based on LBS positioning information of the terminal equipment; and displaying the 3D model resource of the first hidden project on a screen of the terminal equipment in an AR form based on the height information of the terminal equipment. The invention can realize the AR accurate visual early warning of the hidden engineering and improve the safety of the construction process.

Description

AR-based construction indication method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of augmented reality (Augmented Reality, AR for short), in particular to an AR-based construction indication method, an AR-based construction indication device, electronic equipment and a storage medium.

Background

The concealed engineering is mainly a sub-project covered by the subsequent construction, such as foundation, electric pipeline, water supply pipeline, heat supply pipeline, gas pipeline, military optical cable and the like which need to be covered and covered. I.e. a project that is difficult to see again after the construction is completed, such a type of project is called a concealed project. When construction is performed near the position where the hidden engineering is located, in order to avoid damaging the hidden engineering in the construction process, warning is mostly performed by adopting a mode of standing warning boards or pulling warning lines.

However, there are great limitations to such a warning means. Because the marking range is not clear, the specific position, depth and the like of the hidden engineering are difficult to clearly point out, the warning effect in the actual construction process is not obvious, constructors are very easy to touch the hidden engineering because the range of the hidden engineering cannot be clearly pointed out, the hidden engineering is damaged, heavy losses such as engineering reworking are caused, and when serious, gas leakage or electric shock can be caused due to the damage of the hidden engineering, so that the constructors are exposed in potential safety hazards.

Disclosure of Invention

In view of the above, embodiments of the present invention provide an AR-based construction indication method, an AR-based construction indication system, an electronic device, a computer-readable storage medium, and a computer program product, which are used for solving at least one technical problem.

In a first aspect, an embodiment of the present invention provides an AR-based construction indication method, including:

determining UTM position information of the terminal equipment under a UTM coordinate system of a universal transverse ink card support grid system based on GPS position information of the terminal equipment; determining information of one or more hidden projects existing in a set range around the terminal equipment according to UTM position information of the terminal equipment; acquiring 3D model resources of the one or more hidden projects from a preset resource library according to the information of the one or more hidden projects; determining that at least a first concealment project exists in the current visual field range of the terminal equipment based on the current azimuth information of the terminal equipment, wherein the current visual field range of the terminal equipment comprises a region to be constructed, and the one or more concealment projects comprise the first concealment project; determining height information of the terminal equipment based on Location Based Service (LBS) positioning information of the terminal equipment; displaying the 3D model of the first concealment project on a screen of the terminal equipment in an AR form based on the height information of the terminal equipment, wherein the 3D model of the first concealment project is displayed above or below the ground; if the first hidden engineering and the to-be-constructed area are not overlapped, sending out indication information for allowing construction; and if the first hidden engineering is overlapped with the to-be-constructed area, sending alarm information aiming at the overlapped area.

In a second aspect, an embodiment of the present invention provides an AR-based construction indication device, including:

the first determining module is used for determining UTM position information of the terminal equipment under a UTM coordinate system of a universal transverse ink card support grid system based on GPS position information of the terminal equipment;

a second determining module, configured to determine altitude information of the terminal device based on location based service LBS location information of the terminal device;

the third determining module is used for determining information of one or more hidden projects in a set range around the terminal equipment according to UTM position information of the terminal equipment;

the first acquisition module is used for acquiring 3D model resources of the one or more hidden projects from a preset resource library according to the information of the one or more hidden projects;

a fourth determining module, configured to determine, based on current azimuth information of the terminal device, that at least a first concealment project exists in a current field of view of the terminal device, where the current field of view of the terminal device includes a region to be constructed, and the one or more concealment projects include the first concealment project;

the display processing module is used for displaying the 3D model resource of the first hidden project on a screen of the terminal equipment in an AR form; the method comprises the steps of,

The indication processing module is used for sending indication information for allowing construction under the condition that the first hidden engineering and the to-be-constructed area are not overlapped, and sending alarm information for the overlapped area under the condition that the first hidden engineering and the to-be-constructed area are overlapped.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the steps of the method as described above.

In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon computer program instructions which when executed by a processor perform the steps of the method as described above.

In a fifth aspect, embodiments of the present invention provide a computer program product comprising computer program instructions which, when executed by a processor, implement the steps of the method as described above.

By utilizing the AR-based construction indication scheme provided by the embodiment of the invention, the specific position, depth and other information of the hidden engineering in the area to be constructed can be accurately and clearly indicated in the construction process, and the digital model of the hidden engineering is intuitively enhanced and displayed on the position in an AR mode, so that a constructor can conveniently and quickly check the information without complex comparison or calculation, wherein if the hidden engineering is overlapped with the area to be constructed, the current construction range is possibly interfered with the hidden engineering, and the alarm information can be sent to the constructor. Therefore, the invention can realize the early warning of hidden engineering, and intuitively display the early warning result on the screen of the terminal equipment in an AR form, thereby improving the construction safety and the construction efficiency on the whole and reducing the work load of constructors.

Drawings

In order to more clearly describe the technical solution of the embodiments of the present invention, the following description briefly describes the drawings in the embodiments of the present invention.

Fig. 1 is a schematic diagram of an AR system architecture based on a server and a terminal device according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a virtual-real fusion image for AR navigation by using a mobile phone APP.

Fig. 3 is a flowchart of an AR-based construction indication method according to an embodiment of the present invention.

Fig. 4 and 5 are comparative illustrations of the effect of enhancing a concealment engineering 3D model for display above ground using an embodiment of the present invention.

Fig. 6 is a flowchart of a method for converting GPS location information into UTM coordinates according to an embodiment of the present invention.

FIG. 7 is a flow chart of a method of building and using a 3D model repository in accordance with an embodiment of the present invention.

Fig. 8 is a block diagram illustrating a construction indicating device based on AR according to an embodiment of the present invention.

Fig. 9 is a schematic diagram of an electronic device for implementing an AR-based construction indication method according to an embodiment of the present invention.

Fig. 10 is a schematic software architecture of an exemplary terminal device according to an embodiment of the present invention.

Detailed Description

The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It will be appreciated that such embodiments are provided to make the principles and spirit of the invention clear and thorough, and enabling those skilled in the art to better understand and practice the principles and spirit of the invention. The exemplary embodiments provided herein are merely some, but not all embodiments of the invention. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the embodiments herein, are within the scope of the present invention.

Those skilled in the art will appreciate that embodiments of the invention may be implemented as a system, apparatus, device, method, computer readable storage medium, or computer program product. Accordingly, the present invention may be embodied in at least one of the following forms: complete hardware, complete software, or a combination of hardware and software. According to the specific embodiment of the invention, the invention discloses an AR-based construction indication method, an AR-based construction indication system, electronic equipment, a computer-readable storage medium and a computer program product.

In this document, terms such as first, second, third, etc. are used solely to distinguish one entity (or action) from another entity (or action) without necessarily requiring or implying any order or relationship between such entities (or actions).

The embodiment of the invention can be applied to a server and terminal equipment. Referring to fig. 1, a schematic diagram of an AR system architecture based on a server and a terminal device is schematically shown. The AR system architecture comprises a server 10 and several terminal devices 20. In some examples, terminal device 20 is an AR device, which may be a dedicated AR device, such as a Head-mounted AR device (HMD), smart glove, apparel, or other smart wearable electronic device. In some examples, the terminal device 20 may be a general purpose AR device, such as a cell phone, portable computer, notebook computer, tablet computer, virtual Reality (VR) device, vehicle mounted device, navigation device, game device, and the like.

Taking an AR helmet or AR glasses as an example, a head-mounted display, a machine vision system, a mobile computer and the like can be integrated and arranged in a wearable device, the device is provided with a display similar to the glasses in appearance, and is worn on the head of a user in operation, and the device can transmit augmented reality information to the display or projected to eyeballs of the user, so that the visual immersion of the user is enhanced. In some examples, the AR device also has a camera, which may be a wide angle camera, a tele camera, or a structured light camera (also referred to as a point cloud depth camera, a 3D structured light camera, or a depth camera). The structured light camera is based on a 3D vision technology, and can acquire plane and depth information of an object. The structured light camera can project light with certain structural characteristics onto a photographed object through the near infrared laser, then the infrared camera collects reflected light, the reflected light is processed by the processor chip, and the calculation principle is that the position and depth information of the object are calculated according to the change of light signals caused by the object, and a 3D image is displayed. The conventional terminal equipment, such as a mobile phone, presents a two-dimensional image, cannot display the depth of different positions on the image, and can acquire 3D image information data by shooting through a structured light camera, namely, not only can acquire information of colors and the like of different positions in the image, but also can acquire depth information of different positions, and can be used for AR ranging. Of course, the common terminal device can acquire the 2D image based on the optical camera and acquire the depth information of the 2D image by combining a deep learning algorithm and the like, and finally can also present the 3D image.

In some examples, the terminal device 20 has installed therein AR-enabled software or an application APP. The server 10 may be a management server or an application server of the software or APP. The server 10 may be one server, or may be a server cluster formed by a plurality of servers, or may be a cloud server or cloud server. The terminal device 20 has integrated therein a module having a networking function, such as a Wireless-Fidelity (Wifi) module, a bluetooth module, a 2G/3G/4G/5G communication module, etc., so as to be connected to the server 10 through a network.

Illustratively, a user may log into the user account through an APP installed in the cell phone, and the user may also log into the user account through software installed in the AR glasses.

Taking APP with an AR navigation function as an example, the APP may have, for example, a high-precision map navigation capability, an environment understanding capability, a virtual-real fusion rendering capability, and the like, and the APP may report current geographic location information to the server 10 through the terminal device 20, where the server 10 provides an AR navigation service for a user based on the real-time geographic location information. Taking the terminal device 20 as a mobile phone as an example, in response to a user starting an APP, the mobile phone may start a camera to collect an image of a real environment, then perform AR augmentation on the image of the real environment collected by the camera through the system, blend or superimpose a rendered AR effect (such as a navigation route identifier, a road name, merchant information, advertisement presentation, etc.) in the image of the real environment, and present the image of the virtual-real fusion on a screen of the mobile phone.

Fig. 2 schematically shows a virtual-real fusion image of AR navigation by using a mobile phone APP, wherein an indication arrow of AR navigation is superimposed on a real road surface and in a space in the figure, and electronic resources promoted by merchants float at a designated position in the space in the form of a parachute carrying gift box.

Embodiments of the present invention relate to a terminal device and/or a server. Before describing particular embodiments of the present invention, a brief description of technical terms that may be referred to herein will be provided.

GPS (Global Positioning System) is a global positioning system, which is a satellite-based high-precision radio navigation positioning system that provides accurate geographic location, vehicle speed, and accurate time information anywhere in the world and in near-earth space. Because GPS uses longitude and latitude to represent coordinates on the earth, this representation is suitable for describing a coordinate on a three-dimensional surface of the earth.

In the invention, two-dimensional plane coordinates of the terminal equipment are required to be obtained, so that longitude and latitude coordinates of the GPS are required to be converted into UTM coordinates.

UTM (UNIVERSAL TRANSVERSE MERCARTOR GRID SYSTEM) is a universal transverse ink card grid system, and UTM coordinates are rectangular planar coordinates, which have been widely used in topographical maps, typically as a reference grid for satellite images and natural resource databases, and other applications requiring accurate positioning. UTM was formulated by the united states so that the initial zonal is not at the primary meridian, but at 180 degrees so that all united states instincts lie within the 0-30 zone. In a UTM system, first, UTM projection divides the earth surface area between 84 degrees north latitude and 80 degrees south latitude into north-south longitudinal bands, i.e., projection bands, using 6 degree bands. These casting belts are then numbered eastward starting from the 180 degree meridian, i.e. numbered 1 to 60, e.g. Beijing at belt 50. Each band was then divided into 8 degree-weft quadrilaterals, the rows of which start at 80 degrees from south latitude and are marked sequentially with letters C through X (row X includes 72 to 84 degrees total land area from north hemisphere north latitude, 12 degrees total), each quadrilateral being marked with a combination of numbers and letters, and the reference grid read right up.

LBS (Location Based Service) is a location-based service, and LBS is to obtain the current location of a positioning device by using various positioning technologies, and provide information resources and basic services to the positioning device through the mobile internet. The LBS service integrates various information technologies such as mobile communication, internet, space positioning, position information, big data and the like, and utilizes a mobile internet service platform to update and interact data, so that a user can acquire corresponding service through space positioning.

The principles and spirit of the present invention will be explained in detail below by means of several exemplary examples or representative embodiments.

Fig. 3 is a flowchart of an AR-based construction indication method according to an embodiment of the present invention. In this embodiment, the construction indication method includes the steps of:

step S101, determining UTM position information of the terminal device in the UTM coordinate system based on the GPS position information of the terminal device.

Step S102, according to UTM position information of the terminal equipment, determining information of one or more hidden projects existing in a set range around the terminal equipment.

Step S103, according to the information of the one or more hidden projects, acquiring 3D model resources of the one or more hidden projects from a preset resource library.

Step S104, determining that at least a first concealment project exists in the current field of view of the terminal equipment based on the current azimuth information of the terminal equipment, wherein the current field of view of the terminal equipment comprises a region to be constructed, and the one or more concealment projects comprise the first concealment project.

Step S105, determining the height information of the terminal equipment based on the location based service LBS positioning information of the terminal equipment;

step S106, based on the height information of the terminal equipment, displaying the 3D model resource of the first hidden project on a screen of the terminal equipment in an AR form, wherein the 3D model of the first hidden project is displayed above or below the ground;

step S107, if the first hidden engineering and the to-be-constructed area are not overlapped, sending out indication information for allowing construction; and if the first hidden engineering is overlapped with the to-be-constructed area, sending alarm information aiming at the overlapped area.

In embodiments of the present invention, the type of covert engineering may be various above-ground or below-ground engineering facilities that have been or have not been in use, e.g., the covert engineering may be any one or a combination of the following: sewer pipes, tap water pipelines, gas pipelines, power cables, network cables and the like. By utilizing the embodiment of the application, the avoidance protection can be conveniently implemented on the physical facilities hidden underground.

According to the method of the embodiment of the present invention, with respect to step S101, UTM location information of the terminal device in the UTM coordinate system is determined based on GPS location information of the terminal device. The user wears AR glasses or carries a mobile phone provided with an app, and can obtain the current GPS position information of the terminal device. The GPS system adopts longitude and latitude to represent the earth coordinates, namely describes the position coordinates on a three-dimensional earth curved surface, and converts the longitude and latitude coordinates of the GPS into UTM coordinates in the embodiment of the invention, wherein the UTM coordinates are plane rectangular coordinates and are widely applied to a topographic map. In the embodiment of the invention, the two-dimensional plane coordinates of the terminal equipment are obtained by converting the GPS coordinates into UTM coordinates, so as to provide an AR construction instruction for a user.

With respect to step S102, information of one or more concealment projects existing within a set range around the terminal device may be determined according to UTM location information of the terminal device. As one example, a first database is queried based on UTM location information of a terminal device to determine one or more covert engineering identifications. The first database is pre-stored with a mapping relation between UTM position information and hidden engineering identification. The first database may be preset and stored in the server or the terminal device.

Regarding step S103, according to the information of the one or more hidden projects, 3D model resources of the one or more hidden projects are obtained from a preset resource library. As an example, a first repository may be queried based on the one or more covert engineering identifications, and 3D model resources of one or more covert engineering corresponding to the one or more covert engineering identifications may be obtained from the first repository. And the first resource library is pre-stored with hidden engineering 3D model resources corresponding to the hidden engineering identifiers. The first resource library can be pre-constructed and arranged in a server or terminal device.

As an example, risk level labels can be added to the 3D model resources of the hidden engineering according to different mining risk levels, and the labels can be in the form of colors, symbols and characters. In particular, different mining risk levels can be represented by different colors, for example, different types of pipelines can be rendered by different colors, and red, yellow and green represent high risk, medium risk and low risk respectively; and for example, the information of the type, the size, the position and the like of the hidden engineering can be added near the pipeline for the constructors to refer to, so that more accurate and visual guidance is provided for the smooth development of the construction.

With regard to step S104, it is determined that at least a first concealment project exists in the current field of view of the terminal device based on the current azimuth information of the terminal device, where the current field of view of the terminal device includes an area to be constructed, and the one or more concealment projects include the at least a first concealment project. As the concealment engineering may comprise a plurality of different types, for example: sewer, tap water pipeline, gas pipeline, power cable, network cable, etc. Multiple types of concealment projects may be included within the current field of view of the terminal device.

With respect to step S105, altitude information of the terminal device is determined based on the location based service LBS location information of the terminal device. As one example, a user may determine his or her spatial location using LBS location technology, and then the user may obtain location-related resources and information through the mobile internet.

With regard to step S106, the 3D model resource of the first concealment project is displayed in AR form on a screen of the terminal device based on the altitude information of the terminal device, wherein the 3D model of the first concealment project is displayed above or below the ground.

Fig. 4 and 5 are comparative illustrations of the effect of enhancing a concealment engineering 3D model for display above ground using an embodiment of the present invention. Fig. 4 is a schematic diagram of an area to be constructed, which is shot by a mobile phone camera, and it can be seen that the road surface, the vehicle, the space scenery and the like in the area to be constructed in fig. 4 are all conventionally displayed, and no content for enhancing the display exists. FIG. 5 is a schematic view showing the effect of the concealed works after the concealed works are enhanced and displayed in the area to be constructed by using the embodiment of the present invention, it can be seen that a plurality of long pipes (illustratively, hydraulic pipelines and electric pipelines are shown as long pipes indicated by white arrows) appear above the ground of the area to be constructed in FIG. 5, most of the pipes are the same as the extending direction of the road surface, a plurality of the pipes are respectively located at different heights in the space, and pipe branches are also present between some of the pipes; different types of pipes are shown in different colors (treated to remove color in the figure); in addition, the pipeline 3D model is displayed in the space above the road in an AR mode, and the ground shadow formed by the pipeline at the upper side is also displayed on the surface of the road in FIG. 5, so that the effect is vivid.

Thus, constructors can intuitively see the distribution situation of various pipeline works originally hidden below the ground of the current area through the interface shown in fig. 5. The 3D model of the pipeline can be moved to the position below the ground through terminal operation such as gesture instructions, so that the 3D model is positioned at the real height position of the pipeline, more original information can be provided for constructors, and the protection is realized by avoiding the hidden engineering pipeline in the construction area more comprehensively, intuitively and conveniently.

As an example, if there is no overlapping area on the screen of the terminal device, it is indicated that there is no hidden project in the area to be constructed, where construction does not cause damage to the hidden project. If an overlapping area exists on the screen of the terminal equipment, the fact that the hidden engineering exists in the area to be constructed is indicated, the hidden engineering is touched when the construction is performed, damage to the hidden engineering is caused, and dangerous situations such as gas leakage or electric shock are easily caused. Therefore, if the hidden engineering and the area to be constructed are overlapped, the terminal equipment aiming at the overlapped area can timely send out alarm information to constructors so as to prevent accidental injury to the hidden engineering during large-scale excavation.

Optionally, fig. 6 is a flowchart of a method for converting GPS location information into UTM coordinates according to an embodiment of the present invention, and specifically includes the following processing procedures:

step S201, determining an interval number to which the to-be-constructed area belongs in UTM coordinates based on the GPS position information.

Step S202, subtracting the value of the offset corresponding to the interval number of the to-be-constructed area from the UTM coordinate to obtain the X coordinate and the Z coordinate corresponding to the geographic position information of the terminal equipment in the plane rectangular coordinate system.

Step S203, determining the altitude information of the terminal equipment based on the LBS positioning information of the terminal equipment, wherein the altitude position information is the Y coordinate corresponding to the geographic position information of the terminal equipment.

Step S204, setting the position of the terminal equipment as (X, Y, Z) according to the X, Y, Z coordinates in the rectangular coordinate system of the plane of the current position.

Based on this, as an example, converting GPS position information of the terminal device into UTM coordinates may be achieved by:

1) And (3) taking the longitude longitute (deg) of the area to be constructed as 116.351599 and the latitude latitudes (deg) as 39.917891 as parameters, and transmitting the parameters into the formula to obtain the X coordinate UTME (m) value of UTM coordinates as 44757525.244557525 and the Z coordinate UTEN (m) value as 4418875.54467615. It should be noted that if the incoming longitudes and latitudes are in the form of degrees, the conversion needs to be done.

2) The offset value is obtained by calculation of UTM zone block zoneNumber, and the zone block of each zone has its corresponding offset value. The corresponding offset value in this embodiment is (447500, 4418800).

3) Acquiring LBS related to the current position of the terminal equipment through the mobile internet, and determining the height position information of the terminal equipment in the area to be constructed based on the LBS; the altitude position information is Y coordinates corresponding to the geographic position information of the terminal equipment.

4) And subtracting the value of the offset corresponding to the interval number of the region to be constructed from the UTM coordinate to obtain an X coordinate and a Z coordinate corresponding to the geographic position information of the terminal equipment under a plane rectangular coordinate system, namely (-2920.755442475, 75).

5) The terminal device location is set to (-2920.755442475, Y, 75) representing XYZ coordinates of the current location, where Y is the altitude, i.e. the altitude acquired by LBS.

According to the invention, the GPS coordinates are converted into UTM coordinates to obtain the two-dimensional plane coordinates of the terminal equipment of the user, and the height position information of the user is determined by combining with LBS, so that the geographic position information and the height position information of the terminal equipment under a plane rectangular coordinate system are obtained, and preparation is made for providing AR construction instructions for the user. Therefore, the purpose of more accurate positioning is achieved, and the digital model of the hidden engineering is more accurate in the space displayed by the AR equipment in the area to be constructed. FIG. 7 is a flow chart of a method of building and using a 3D model repository in accordance with an embodiment of the present invention. In this embodiment, two correlation libraries are included: a first database and a first repository. And the first database is pre-stored with a mapping relation between UTM position information and hidden engineering identification. And the first resource library is pre-stored with hidden engineering 3D model resources corresponding to the hidden engineering identification, wherein the hidden engineering 3D model resources comprise information such as model resource rotation, size, position, color, model file resource position and the like.

As shown in fig. 5, the method for establishing the 3D model resource library includes the following steps:

step S301, a basic 3D model file is built in and stored in a first resource library. Wherein the base 3D model file includes at least one of: a foundation 3D model of sewer, tap water pipeline, gas pipeline, power cable, network cable, etc.

Step S302, data information of the area to be constructed is stored in a first database, and relevant position information is calculated. And storing the information into a corresponding first database according to the data information edited on site in the area to be constructed or the information analyzed according to the CAD graph, and then calculating the position information through indexes. Wherein the field edited data information includes the size, position, type of the 3D model to be used, etc. of the desired 3D model.

Step S303, calculating the position change in real time through SLAM of the terminal equipment. The azimuth of the 3D model is adjusted or displayed through rotation and movement, and real-time rendering display is carried out on the 3D model, so that the hidden engineering 3D model of the area to be constructed can be more real. Application fields of SLAM (Simultaneouslocalization and mapping) include the fields of robot positioning and navigation, unmanned aerial vehicle, VR/AR, etc. The method is mainly used for assisting in enhancing visual effect in VR/AR. The SLAM technology can construct a map with a more real visual effect, so that the superposition effect of the virtual object is rendered aiming at the current visual angle, and the virtual object is more real and has no offensive sense. The SLAM system framework is generally divided into five modules, including sensor data, a visual odometer, a back end, map building and loop detection. The sensor data are mainly used for collecting various types of original data in an actual environment, including laser scanning data, video image data, point cloud data and the like. The visual odometer is mainly used for estimating the relative positions of moving targets at different moments, and comprises the application of algorithms such as feature matching and direct registration. The back end is mainly used for optimizing accumulated errors brought by the visual odometer, and comprises algorithm applications such as a filter, graph optimization and the like. The map is mainly used for three-dimensional map construction. Loop detection is mainly used for spatial cumulative error cancellation. Illustratively, in this embodiment, the workflow is:

1) And collecting various types of original data in the actual environment through the sensor of the terminal equipment.

2) After the sensor reads the data, the visual odometer estimates the relative motion (Ego-motion) at both moments.

3) The rear end processes the estimated result of the visual odometer and optimizes the accumulated error brought by the estimated result.

4) And (3) establishing a map according to the motion trail obtained by the front end and the rear end, namely, establishing a map process.

5) Loop detection takes into account images at different moments of the same scene, providing spatial constraints to eliminate accumulated errors.

In addition, as an example, in some embodiments of the present invention, the area to be constructed range may be determined by:

1) The current location of the terminal device is set to a dot. For example, the current position is P ₁ Its plane coordinate corresponds to P ₁ ＝(x ₁ ，y ₁ ). The position range of the hidden engineering 3D model resource needs to be displayed, wherein the plane coordinate of the central point where the position range is positioned corresponds to P ₂ ＝(x ₂ ，y ₂ )。

2) And defining an area according to the radius set by the user to serve as an area to be constructed, wherein the set range around the terminal equipment is larger than or equal to the area to be constructed. According to P ₁ And P ₂ Plane coordinates of (d) can be obtained and set to the range d ₁₂ Is defined by the relation:

and then P can be obtained ₂ Is defined by the plane coordinates of:

for example, the current location is (-2920, 75), the location range of the display hidden engineering 3D model is 500 meters, then the location of the display model should be at:

-3420<x ₂ <-2420，-425<y ₂ <575

the hidden engineering 3D model resources in the range are displayed. In the actual construction process, d can be adjusted according to the field requirement ₁₂ Setting is performed.

Step S304, information of hidden projects existing in the target area is obtained. The method for collecting the space position information and the hidden engineering type information of the hidden engineering existing in the target area comprises the following steps: according to digital resources corresponding to the hidden projects, acquiring space position information and hidden project type information of the hidden projects in the target area in batches; or using the image acquisition equipment with positioning capability to acquire the image of the position of the hidden project in the target area, receiving the space position information of the hidden project and the information of the hidden project type input by a user, and displaying the space position occupied by the hidden project on a screen of the image acquisition equipment.

Step S305, loading the data information of the hidden engineering in the target area. And loading the data information of the hidden engineering into the APP of the terminal equipment.

And step S306, displaying the hidden engineering 3D model of the area to be constructed in the AR form on the terminal equipment.

In some embodiments, risk level labels can be added to 3D model resources of hidden projects of areas to be constructed according to different mining risk levels, and the label form comprises at least one of the following: color, symbol, text. The terminal equipment can be AR glasses or mobile phones and the like. As shown in fig. 7 (color is not shown), the terminal device displays the hidden engineering 3D model of the area to be constructed through different colors, for example, red is used for indicating that the mining risk level is the highest, yellow is used for indicating that the mining risk level is the next, green is used for indicating that the mining risk is lower, and the like. Meanwhile, different types of character labeling pipelines can be used as auxiliary materials, or the mining risk level can be prompted by directly using character labeling forms such as 'high risk', 'inflammable', 'explosive'. In other embodiments, the indication of danger may also be provided directly, e.g. by using a symbol indicating the danger

Equal sign. The mining risk of some places is represented by different colors, or different pipeline information is displayed by using different colors, so that more accurate and visual guidance can be provided for construction.

In other embodiments, the 3D model of the covert project may be displayed in AR form on a corresponding location of the real sink space by performing a perspective or translucency process on the real sink space. Through the display mode, the digital model of the hidden engineering can be intuitively enhanced and displayed on the position in an AR mode, namely the 3D model is positioned at the position of the hidden engineering, so that constructors can intuitively know the occupied area of the hidden engineering, and complex comparison or calculation is not needed.

In the embodiment of the application, for example, according to gesture operation of a user, the 3D model of the hidden engineering can be dynamically displayed in a lifting mode and a descending mode, so that the 3D model is located at different vertical heights, and the hidden engineering is convenient for the user to check according to the requirement. Furthermore, it is possible to implement a display in which one or more concealed engineering types are selected, for example only a 3D model of "gas pipeline" is displayed, and for example also a 3D model of "network cable" and "power cable" is displayed simultaneously.

Specific implementations of the construction indication method of the embodiments of the present application are described above through a plurality of embodiments. Based on the embodiment of the application, a constructor opens terminal equipment in an area to be constructed, such as AR glasses or mobile phones and other terminal equipment, the terminal equipment can locate the GPS position of a user, the current height is determined by converting the GPS position into UTM coordinates and then combining with an LBS technology, the user position information is sent to a server, and the server subtracts the offset under the UTM and can convert the offset to obtain hidden engineering position information stored in a search library. And then, the server searches the hidden engineering model resource information in a specified range around the current position at the server side according to the position information, and downloads the hidden engineering model resource information into the terminal equipment. The terminal equipment can determine the specific direction in front of the camera through the gyroscope, set the coordinate position of the AR camera according to the position information returned by the server, load the model resources of underground hidden projects in the surrounding designated range to the designated position returned by the server, enable the model position of the hidden projects to be matched with the real position, and display the model resources in an AR mode. In the display process, the hidden engineering model can be sunk from the current height to a real position below the ground through gesture instructions, and the model can be moved to the position above the ground again to facilitate observation.

Thus, after the constructor wears the AR glasses or carries the mobile phone for installing the app, the constructor can clearly see the information such as the specific position, the extension depth, the breadth and the like of the hidden engineering in the to-be-constructed area through the terminal equipment in the construction process. Further, when the condition that the hidden engineering is overlapped with the area to be constructed is detected, the terminal equipment can send alarm information to constructors. Therefore, the invention can realize the early warning of the hidden engineering and intuitively display the early warning result on the screen of the terminal equipment in an AR form, thereby avoiding damage to the existing hidden engineering in the actual construction process.

Further, for clarity, as an example, the following briefly describes an algorithmic process for converting GPS location information into UTM coordinates. Assume that the longitude and latitude of a certain point in space are respectively

The UTM coordinates are expressed as (E, N) in radians, km in km, and the UTM banded calculation process is as follows:

zoneNumber= (λ (gap)/6) round +31 (UTM area block)

λ ₀ ＝(zoneNumber-1)*6-180+3(degree)

k ₀ ＝0.9996,E ₀ ＝500km,e＝0.0818192

N ₀ ＝0km(Northern hemispher)or 10000km(Southern hemisphere)

For example, the UTM band number range spanned by the Chinese national border is 43-53, and the China territory range is specifically as follows:

39 degrees north latitude at the most west and 73 degrees 33 degrees east longitude;

53.5 min north latitude at the north end and 27 min 124 th east longitude;

3 degrees 51 min for north latitude and 112 degrees 16 min for east longitude;

47.5 minutes north latitude at the eastern end and 46.5 minutes 134 degrees east longitude.

As an example, taking the Guizhou province as an example, the UTM cast tape number of the Guizhou province is calculated as follows:

1) Guizhou province is located in the northern hemisphere, selecting the last letter "N" band.

2) Calculated according to the above formula, the integer part +31 of the band number= (longitude integer/6).

The central coordinates of Guizhou province are about latitude 106.709177096,26.6299067414 and longitude 106.709177096, and the number of bands in which the central coordinates are located is calculated:

banding number= 106.709177096/6+31=48

Namely, the number of bands in Guizhou province is: 48N.

Corresponding to the above method embodiment, the present invention further provides an AR-based construction indication device, and fig. 8 is a block diagram of an AR-based construction indication device 100 according to an embodiment of the present invention. The AR-based construction indication device 100 includes:

a first determining module 110, configured to determine UTM position information of the terminal device under a UTM coordinate system of a universal transverse ink card grid system based on GPS position information of the terminal device;

a second determining module 120, configured to determine altitude information of the terminal device based on location based service LBS location information of the terminal device;

A third determining module 130, configured to determine, according to UTM position information of the terminal device, information of one or more hidden projects existing in a set range around the terminal device;

a first obtaining module 140, configured to obtain 3D model resources of the one or more hidden projects from a preset resource library according to the information of the one or more hidden projects;

a fourth determining module 150, configured to determine, based on current azimuth information of the terminal device, that at least a first concealment project exists in a current field of view of the terminal device, where the current field of view of the terminal device includes a region to be constructed, and the one or more concealment projects include the first concealment project;

a display processing module 160, configured to display the 3D model resource of the first concealment engineering on a screen of a terminal device in an AR form;

the indication processing module 170 is configured to send indication information for allowing construction when the first concealed project and the to-be-constructed area do not overlap, and send alarm information for an overlapping area when the first concealed project and the to-be-constructed area overlap.

The embodiment of the application also provides an AR-based construction indication system, which comprises AR terminal equipment and a server, wherein the AR terminal equipment comprises the construction indication device 100 shown in fig. 8. The server includes a server communication module, a resource providing module 202, and a processing module. The server communication module is used for communicating with the AR terminal equipment, and transmitting data, information and the like between the server and the AR terminal equipment. The resource providing module is connected with the server communication module, receives the current position information to be constructed from the AR terminal equipment, and returns the 3D model resource information corresponding to the position to be constructed to the AR terminal equipment. The resource providing module further comprises a first database and a first resource library. And the first database is pre-stored with a mapping relation between UTM position information and hidden engineering identification. And the first resource library is pre-stored with hidden engineering 3D model resources corresponding to the hidden engineering identifiers. The first repository 2021 is queried according to the one or more covert engineering identifications to obtain 3D model resources of the one or more covert engineering corresponding to the one or more covert engineering identifications from the first repository. The processing module is respectively connected with the server communication module and the resource providing module, so as to respond to the requests of the server communication module and the resource providing module, process related data and send the related data to a display terminal (not labeled in the figure) so as to display the 3D model on the display terminal.

According to the method and the system provided by the embodiment of the invention, the AR-based construction indication scheme can solve the problems that the indication range of the hidden engineering is not clear in the construction process, the specific position, depth and the like of the hidden engineering are difficult to clearly indicate, so that the problem that the warning effect is not obvious in the actual construction process, constructors can not clearly indicate the range of the hidden engineering to touch the hidden engineering, damage to the hidden engineering is caused, heavy losses such as engineering reworking are caused, and even gas leakage or electric shock is caused due to damage to the hidden engineering, so that the constructors are exposed in potential safety hazards is solved.

Those skilled in the art will appreciate that the embodiments described herein are presently preferred embodiments, and that the acts, steps, modules, or units, etc. that are involved are not necessarily required by embodiments of the invention. In the foregoing embodiments, the descriptions of the embodiments of the present invention are emphasized, and in part, not described in detail in one embodiment, reference may be made to related descriptions of other embodiments.

Fig. 9 is a schematic structural diagram of an electronic device 10 according to an embodiment of the present invention, where the electronic device 10 includes a processor 11, a memory 12, and a communication bus for connecting the processor 11 and the memory 12, and a computer program that can be run on the processor 11 is stored in the memory 12, and when the processor 11 runs the computer program, the steps in the method according to the embodiments of the present invention are executed or called implemented. The electronic device 10 may be a server in the embodiment of the present invention, and the electronic device 10 may also be a cloud server. The electronic device 10 may also be a terminal device or an AR device in an embodiment of the present invention. Electronic device 10 may also be a cloud server. The electronic device 10 also includes a communication interface for receiving and transmitting data.

In some embodiments, processor 11 may be a Central processor (Central ProcessingUnit, CPU), a graphics processor (graphics processing unit, GPU), an application processor (application processor, AP), a modem processor, an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, a neural-network processor (neural-network processing unit, NPU), or the like; the processor 11 may also be other general purpose processors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor or the like. The NPU can rapidly process input information and can continuously perform self-learning by referring to the biological neural network structure. Applications such as intelligent recognition, image recognition, face recognition, semantic recognition, voice recognition, text understanding, etc. can be implemented by the NPU electronic device 10.

In some embodiments, the memory 12 may be an internal storage unit of the electronic device 10, such as a hard disk or memory of the electronic device 10; the memory 12 may also be an external storage device of the electronic device 10, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, which are provided on the electronic device 10. Memory 12 may also include both internal storage units and external storage devices of electronic device 10. The memory 12 may be used to store an operating system, application programs, boot loader (BootLoader), data, and other programs, such as program code for computer programs, and the like. Memory 12 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasableprogrammable read only memory, EPROM), or portable read-only memory (CD-ROM). Memory 12 is used to store program codes executed by electronic device 10 and transmitted data. The memory 12 may also be used to temporarily store data that has been output or is to be output.

It will be appreciated by those skilled in the art that fig. 9 is merely an example of the electronic device 10 and is not intended to limit the electronic device 10, and that the electronic device 10 may include more or less components than illustrated, or may combine certain components, or may include different components, such as may also include input-output devices, network access devices, and the like.

Fig. 10 is a schematic software structure of a terminal device according to an embodiment of the present invention. Taking a mobile phone operating system as an Android system as an example, in some embodiments, the Android system is divided into four layers, which are respectively: an application layer, an application framework layer (FWK), a system layer, and a hardware abstraction layer, the layers communicating via a software interface.

First, the application layer may include a plurality of application packages, which may be various application apps such as call, camera, video, navigation, weather, instant messaging, education, etc., or may be application apps based on AR technology.

Second, the application framework layer FWK provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer may include some predefined functions, such as functions for receiving events sent by the application framework layer.

The application framework layer may include a window manager, a resource manager, a notification manager, and the like.

Wherein the window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like. The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

Among other things, the resource manager provides various resources to the application, such as localization strings, icons, pictures, layout files, video files, and so forth.

The notification manager enables the application program to display notification information in a status bar, can be used for conveying notification type information, and can automatically disappear after a short stay without user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

In addition, the application framework layer may also include a view system that includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views, for example, a text display view and a picture display view may be included on the display interface of the sms notification icon.

Third, the system layer may include a plurality of functional modules, such as a sensor service module, a physical state recognition module, a three-dimensional graphics processing library (e.g., openGLES), and so forth.

The sensor service module is used for monitoring sensor data uploaded by various sensors of the hardware layer and determining the physical state of the mobile phone; the physical state recognition module is used for analyzing and recognizing gestures, faces and the like of a user; the three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

In addition, the system layer may also include a surface manager and a media library. The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications. Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like.

Finally, a hardware abstraction layer is a layer between hardware and software. The hardware abstraction layer may include display drivers, camera drivers, sensor drivers, etc. for driving the relevant hardware of the hardware layer, such as the display screen, camera, sensor, etc.

The embodiments of the present invention also provide a computer-readable storage medium storing a computer program or instructions which, when executed, implement the steps in the method designed in the above embodiments.

Embodiments of the present invention also provide a computer program product comprising a computer program or instructions which, when executed, implement the steps in the method devised in the embodiments described above. The computer program product may be, for example, a software installation package.

Those skilled in the art will appreciate that the methods, steps, or functions of related modules/units described in the embodiments of the present invention may be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product, or in the form of computer program instructions executed by a processor. Wherein the computer program product comprises at least one computer program instruction, which may be comprised of corresponding software modules, which may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, a removable disk, a compact disk read-only memory (CD-ROM), or any other form of storage medium known in the art. The computer program instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer program instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center, by wire or wirelessly. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), optical medium, or semiconductor medium (e.g., SSD), etc.

With respect to each of the apparatuses/products described in the above embodiments, the modules/units included therein may be software modules/units, or may be hardware modules/units, or may be partly software modules/units, or partly hardware modules/units. For example, for an application or a device/product integrated on a chip, each module/unit included in the device/product may be implemented in hardware such as a circuit, or at least some modules/units may be implemented in software programs, and run on a processor integrated inside the chip, where the remaining modules/units are implemented in hardware such as a circuit. For another example, for an application or a device/product integrated in a terminal, each module/unit included in the device/product may be implemented in hardware such as a circuit, or at least some modules/units may be implemented in software program, and run on a processor integrated in the terminal, where the rest of modules/units may be implemented in hardware such as a circuit.

In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present invention is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and they should be included in the scope of the present invention.

Claims

1. An AR-based construction indication method, comprising:

determining UTM position information of the terminal equipment under a UTM coordinate system of a universal transverse ink card support grid system based on GPS position information of the terminal equipment;

determining information of one or more hidden projects existing in a set range around the terminal equipment according to UTM position information of the terminal equipment;

acquiring 3D model resources of the one or more hidden projects from a preset resource library according to the information of the one or more hidden projects;

determining that at least a first concealment project exists in the current visual field range of the terminal equipment based on the current azimuth information of the terminal equipment, wherein the current visual field range of the terminal equipment comprises a region to be constructed, and the one or more concealment projects comprise the first concealment project;

determining height information of the terminal equipment based on Location Based Service (LBS) positioning information of the terminal equipment;

displaying the 3D model of the first concealment project on a screen of the terminal equipment in an AR form based on the height information of the terminal equipment, wherein the 3D model of the first concealment project is displayed above or below the ground; and

If the first hidden engineering does not overlap with the to-be-constructed area, sending out indication information allowing construction; and if the first hidden engineering is overlapped with the to-be-constructed area, sending alarm information aiming at the overlapped area.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

displaying the 3D model of the first hidden project at the height of the terminal equipment; and/or

And displaying the 3D model of the first concealment engineering below the perspective-processed ground, wherein the height of the 3D model is consistent with that of the first concealment engineering.

3. The method of claim 1, wherein determining information of one or more concealment projects existing within a set range around the terminal device according to UTM location information of the terminal device comprises:

querying a first database based on UTM (universal time measurement) position information of terminal equipment to determine one or more hidden engineering identifications; the first database is pre-stored with a mapping relation between UTM position information and hidden engineering identification.

4. The method of claim 3, wherein obtaining 3D model resources of the one or more covert projects from a preset resource library based on information of the one or more covert projects, comprises:

Querying a first resource library according to the one or more hidden engineering identifiers to obtain 3D model resources of one or more hidden engineering corresponding to the one or more hidden engineering identifiers from the first resource library; and the first resource library is pre-stored with hidden engineering 3D model resources corresponding to the hidden engineering identifiers.

5. The method as recited in claim 4, further comprising:

collecting space position information of hidden projects existing in a target area and hidden project type information;

based on the space position information and the hidden engineering type information of the hidden engineering, generating a 3D model resource of the hidden engineering, and adding an identifier to the generated 3D model resource;

constructing a first database based on the space position information of the hidden engineering and the corresponding 3D model resource identifier; and

and constructing a first resource library based on the 3D model resource identification of the hidden engineering and the corresponding 3D model resource.

6. The method of claim 5, wherein the collecting spatial location information and concealment engineering type information for concealment engineering present within the target area comprises:

according to digital resources corresponding to the hidden projects, acquiring space position information and hidden project type information of the hidden projects in the target area in batches; and/or the number of the groups of groups,

And acquiring an image of the position of the hidden project in the target area by using an image acquisition device with positioning capability, receiving the space position information of the hidden project and the information of the hidden project type input by a user, and displaying the space position occupied by the hidden project on a screen of the image acquisition device.

7. The method as recited in claim 1, further comprising:

and the current position of the terminal equipment is used as a round point, an area is defined according to the radius set by a user to serve as an area to be constructed, and the set range around the terminal equipment is larger than or equal to the area to be constructed.

8. The method as recited in claim 1, further comprising:

adding risk level labels to 3D model resources of hidden projects according to different mining risk levels, wherein the label forms comprise at least one of the following: color, symbol, text.

9. The method of any of claims 1-8, wherein the type of concealment engineering comprises at least one of: sewer, tap water pipeline, gas pipeline, power cable, network cable.

10. An AR-based construction indicating device, comprising:

the display processing module is used for displaying the 3D model resource of the first hidden project on a screen of the terminal equipment in an AR form; and

11. An electronic device, comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method of any of claims 1-9.

12. A computer readable storage medium, characterized in that the computer storage medium has stored thereon computer program instructions which, when executed by a processor, implement the method according to any of claims 1-9.

13. A computer program product comprising computer program instructions which, when executed by a processor, implement the method of any one of claims 1-9.