KR102484280B1 - System for providing adjacent building pre-survey service usign drone - Google Patents

Abstract

Provided is a system for providing a survey service around a structure using a drone, which comprises: a drone capturing a location from the sky in which at least one event requiring a survey around a structure occurs and streaming the same in real time to compare before and after occurrence of at least one event; a 360-degree VR camera capturing 360 degrees horizontally and vertically to create a spherical photo or video; a user terminal overlaying damage data, including a location, type, size, length, and shape of damage extracted and labeled within the photo or video received from the 360-degree VR camera, onto the photo or video, and synchronizing real-time drone data streamed from the drone with the damage data to output the same; and a survey service providing server including a complanation unit which performs complanation by stitching the spherical photo or video received from the 360-degree VR camera, an extraction unit which extracts the damage data containing the location, type, size, length, and shape of the damage from the spherical photo or image, a transmission unit which transmits the damage data to the user terminal, and a synchronization unit which synchronizes the time of the damage data and real-time drone data and outputs the damage data and real-time drone data. Accordingly, GPS coordinates for an object and changes in a state of buildings or facilities before, during, and after construction can be identified.

Description

Year-end survey service using drone {SYSTEM FOR PROVIDING ADJACENT BUILDING PRE-SURVEY SERVICE USIGN DRONE}

The present invention relates to a system for providing year-end survey services using drones, and provides a system capable of establishing absolute standards for investigation of construction damage before and after the start of construction by synchronizing images taken using a drone and a 360-degree camera. .

Recently, due to the necessity of expanding social infrastructure in urban areas, the trend of constructing tunnels for railroads and subways is gradually increasing. Because urban tunnels are often built on relatively shallow floors, ground displacement is emerging as a major factor governing tunnel design. In particular, one of the essential requirements for urban tunnel construction is to secure the safety of the underground and various adjacent structures existing on the ground. There are many cases of their victims. One of the essential conditions to be considered in the design of tunnels built in the shallow urban area is not to cause excessive damage to adjacent structures. At this time, a pre- or post-survey is conducted along the roadside, and the damage value is determined by setting the range of influence according to the type of construction targeting adjacent buildings that are judged to have an impact on the vicinity of the construction site. and damage progression will be investigated.

At this time, laser scanning is used when pre-surveying the degree of synapse, or it is made by comparing vibration values after the fact. Prior art, Korean Patent Publication No. 2011-0127517 (published on November 25, 2011) and Korea Patent Publication In No. 2007-0077218 (published on July 25, 2007), laser scanning of the cross-section of the ground is performed every time a predetermined depth is dug in the ground excavation construction stage for building construction to investigate the accurate geological condition, In order to supplement and change the design by providing feedback on the results, reference points are arranged and installed at the geological survey site, laser scanning of the ground and reference points is performed, the laser-scanned ground is photographed with a digital camera, and excavation work is performed. As the process progresses, after repeating laser scanning and digital camera shooting, 3D image data for the ground are generated, and the generated image data is analyzed to generate data necessary for building design, and adjacent buildings according to subway operation In order to measure the safety of the subway, a configuration for measuring and monitoring the safety of adjacent buildings by measuring subway acceleration signals, converting the acceleration signals into speed or displacement signals, and comparing and analyzing with standard subway vibration values is disclosed.

However, in the case of the former, when building a building, feedback is received and reflected on what affects the building itself, and environmental impact assessment on adjacent buildings is not carried out. It is not a configuration that measures cracks in the building itself. A method of preventing disputes and reducing social costs to zero has been developed by using a 360-degree camera to survey and draw the riverside with 3D stereoscopic images, but this method is also limited to pre-surveying the roadside and changes during construction in real time. is not a configuration that can be monitored. Therefore, it is required to research and develop a system that surveys and draws the roadside with 3D stereoscopic images using a 360-degree camera, and synchronizes and provides real-time drone data photographed using a drone with damage data.

An embodiment of the present invention extracts damage data including the location, type, size, length and shape of damage from a photo or video received from a 360 degree VR camera, and synchronizes the time of real-time drone data streamed from a drone. It enables annual surveys according to real-time construction events, and GPS coordinates for objects captured from real-time drone data using GPS, and changes in building or facility conditions before, during, and after construction can be grasped, so it is not limited to exterior surveys. It is possible to provide a roadside survey service providing system using drones that can proceed to internal survey without doing so. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention, photographing a place where at least one event for which annual survey is required occurs from the air, and before and after at least one event occurs. Drones streaming in real time to compare damage, 360 degree VR cameras taking 360 degrees horizontally and vertically to create spherical pictures or videos, locations, types of damage extracted and labeled within pictures or videos received from 360 degree VR cameras, The spherical surface received from the user terminal and 360-degree VR camera that overlays damage data, including size, length, and shape, on a photo or video, and synchronizes and outputs real-time drone data streamed from the drone with the damage data A flattening unit that performs complanation by stitching photos or images, an extraction unit that extracts damage data including location, type, size, length, and shape of damage from spherical photos or images, and damage data to user terminals and a survey service providing server including a transmission unit that transmits and a synchronization unit that synchronizes time between damage data and real-time drone data and outputs damage data and real-time drone data.

According to any one of the above-described problem solving means of the present invention, damage data including the location, type, size, length, and shape of damage is extracted from a photo or video received from a 360-degree VR camera, and the real-time drone streamed from the drone By synchronizing the time of the data, it is possible to investigate yearly changes according to real-time construction events, and by using GPS, it is possible to grasp the GPS coordinates of objects captured from real-time drone data and changes in building or facility conditions before, during, and after construction. It is not limited to exterior inspections, but can also proceed with interior inspections.

1 is a diagram for explaining a roadside survey service providing system using a drone according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a survey service providing server included in the system of FIG. 1 .
3 and 4 are views for explaining an embodiment in which a roadside survey service using a drone according to an embodiment of the present invention is implemented.
5 is an operation flowchart for explaining a method of providing a roadside survey service using a drone according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

As used throughout the specification, the terms "about", "substantially", etc., are used at or approximating that value when manufacturing and material tolerances inherent in the stated meaning are given, and do not convey an understanding of the present invention. Accurate or absolute figures are used to help prevent exploitation by unscrupulous infringers of the disclosed disclosure. The term "step of (doing)" or "step of" as used throughout the specification of the present invention does not mean "step for".

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. On the other hand, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or further separated into additional components and '~units'. In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, device, or device may be performed instead by a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data. can be interpreted as

One embodiment of the present invention is an improved patent of Korea Patent Registration No. 10-2388777 (published on April 21, 2022), which is a prior registration patent of the applicant, and the same content as disclosed in the prior registration patent is the same as the above-mentioned Korean Registered Patent No. 10-2388777 (announced on April 21, 2022) and will not be described in detail herein. However, just because it is not described in this specification does not mean that it is not included in an embodiment of the present invention, and it is specified that it is not described in order to avoid duplication.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a roadside survey service providing system using a drone according to an embodiment of the present invention. Referring to FIG. 1 , the drone-based survey service providing system 1 includes at least one user terminal 100, a survey service providing server 300, at least one 360-degree VR camera 400, and at least one may include a drone 500 of However, since the roadside survey service providing system 1 using the drone of FIG. 1 is only one embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to a research service providing server 300 through a network 200 . In addition, the survey service providing server 300 may be connected to at least one user terminal 100, at least one 360 degree VR camera 400, and at least one drone 500 through the network 200. In addition, at least one 360 degree VR camera 400 may be connected to the survey service providing server 300 through the network 200 . In addition, at least one drone 500 may be connected to at least one user terminal 100, the survey service providing server 300, and at least one 360 degree VR camera 400 through the network 200.

Here, the network refers to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers, and examples of such networks include a local area network (LAN) and a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communications networks, telephone networks, and wired and wireless television communications networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi , Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( A Near-Field Communication (Near-Field Communication) network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, etc. are included, but not limited thereto.

In the following, the term at least one is defined as a term including singular and plural, and even if at least one term does not exist, each component may exist in singular or plural, and may mean singular or plural. It will be self-evident. In addition, the singular or plural number of each component may be changed according to embodiments.

At least one user terminal 100 uses a web page, app page, program, or application related to a roadside survey service using a drone to display a photo or video taken of a roadside facility, building, etc. in a state in which damage data is overlaid. It may be a user's terminal that outputs 3D or 2D drawings. In addition, the user terminal 100 synchronizes and outputs a picture or video overlaid with the damage data with the video of the real-time drone data of the drone 500, and outputs the result of identifying the GPS location in the real-time drone data in the real-time drone data. It may be a terminal that

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

The survey service providing server 300 may be a server that provides an annual survey service web page, app page, program or application using drones. In addition, the investigation service providing server 300 may be a server that models at least one artificial intelligence algorithm by learning and testing to distinguish at least one type of damage. At this time, when the spherical photo or image is received from the 360 degree VR camera 400, the survey service providing server 300 creates a 2D or 3D drawing after stitching and flattening, and extracts the damage data on the image frame or image After classifying and labeling, it may be a server that transmits to the user terminal 100 . In addition, the survey service providing server 300 may be a server that outputs real-time drone data transmitted from the drone 500 in synchronization with damage data. At this time, the meaning of synchronizing the time of the real-time drone data with the damage data may mean that the time of a photo or video overlaid with the damage data is synchronized and output.

Here, the survey service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser.

At least one 360-degree VR camera 400 is connected to the user terminal 100 with or without using a web page, app page, program, or application related to the year-end survey service using drones, and the survey service providing server 300 It may be a device that transmits spherical images or photos.

Here, at least one 360-degree VR camera 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one 360-degree VR camera 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one 360-degree VR camera 400 is, for example, a wireless communication device that ensures portability and mobility, and includes navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular) ), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless It may include all kinds of handheld-based wireless communication devices such as Broadband Internet terminals, smartphones, smart pads, tablet PCs, and the like.

At least one drone 500 uses a web page, app page, program, or application related to annual survey service using drones to identify and mark the location of each facility or building with GPS, and provides real-time drone data to the survey service providing server It may be a device that transmits to (300).

Here, at least one drone 500 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one drone 500 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one drone 500 is, for example, a wireless communication device that ensures portability and mobility, and includes navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

2 is a block diagram illustrating a survey service providing server included in the system of FIG. 1, and FIGS. 3 and 4 are an embodiment of an annual survey service using a drone according to an embodiment of the present invention. It is a drawing for explaining.

Referring to FIG. 2 , the survey service providing server 300 includes a flattening unit 310, an extraction unit 320, a transmission unit 330, a synchronization unit 340, an interlocking unit 350, and a location identification marking unit 360. ), an automatic enlargement unit 370, a remote control unit 380, a comparison screen providing unit 390, and an artificial intelligence unit 391 may be included.

The investigation service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with at least one user terminal 100, at least one 360 degree VR camera 400, and at least one In the case of transmitting an annual survey service application, program, app page, web page, etc. using a drone 500, at least one user terminal 100, at least one 360 degree VR camera 400, and at least one The drone 500 may install or open an annual survey service application, program, app page, web page, etc. using the drone. In addition, the service program may be driven in at least one user terminal 100, at least one 360 degree VR camera 400, and at least one drone 500 by using a script executed in a web browser. Here, the web browser is a program that allows users to use the web (WWW: World Wide Web) service, and means a program that receives and displays hypertext described in HTML (Hyper Text Mark-up Language). For example, Netscape , Explorer, Chrome, and the like. In addition, an application means an application on a terminal, and includes, for example, an app running on a mobile terminal (smart phone).

Referring to FIG. 2 , the planarization unit 310 may perform complanation by stitching a spherical photo or image received from the 360 degree VR camera 500 . The 360-degree VR camera 500 may create a spherical photo or video by taking 360-degree horizontal and vertical images. At this time, it is possible to use a method of converting the image itself to create a mask only with brightness information, and to reduce the unnaturalness of the seam and reduce the brightness error between images, a Voronoi boundary extraction method using a mask extraction (seam finding) method, A dynamic programming method can be used, and a graph cut method, a method of extracting a minimum error boundary between two images, and multi-angle image synthesis can be used. As described above, since it is disclosed in detail in the prior registration patent of the present applicant, duplicate contents are omitted.

The extraction unit 320 may extract damage data including location, type, size, length, and shape of damage from a spherical photograph or image. The synchronization unit 330 may output damage data and real-time drone data by synchronizing the times of the transmission unit that transmits the damaged data to the user terminal 100 and the damaged data and the real-time drone data. The drone 500 may photograph a place where at least one event requiring annual survey occurs from the air, and perform real-time streaming to compare before and after the occurrence of the at least one event. For example, at least one event may be a blasting event. The user terminal 100 overlays damage data, including the location, type, size, length, and shape of the damage extracted and labeled in the photo or image received from the 360-degree VR camera 500, on the photo or image. ), and the real-time drone data streamed from the drone 500 may be synchronized with the damage data and output.

Recently, drones, which are unmanned aerial vehicles (UAVs), are becoming increasingly useful as one of the major industries of the 4th industrial revolution, and drone photogrammetry is used to acquire GPS. This is a method for acquiring location information during shooting by mounting a sensor such as a camera, GPS, or INS (Inertial Navigation System) on a drone and generating 3D spatial information through image processing.

<Ground Control Point Survey>

To generate spatial information with accurate location information, ground control point (GCP) surveying can be performed first. By matching the location information obtained through this with the coordinate values of the image taken by the drone, more precise surveying is possible. Anti-aircraft beacons can be manufactured and installed at each site, and for example, the coordinates of the ground control point can be acquired by using the RTK-GPS (Real Time Kinematic-Virtual Reference Station) method using ComNav's T30 IMU GNSS RECEIVER.

<GPS Survey>

It is possible to further use a GPS surveying research device, and for data collection, the user uses the GPS of the user terminal 100, and the track record can be obtained by using only the built-in GPS of the smartphone without the investigator connecting a separate external device. . The area can be calculated using the AutoCAD program from the track record obtained by the investigator wearing the surveying device and walking the event boundary.

<RTK-GPS Survey>

RTK-GPS survey can use ComNav's T30 IMU GNSS RECEIVER. It is a VRS (Virtual Reference Station)-RTK survey method that enables RTK survey with one mobile station GPS without using the reference station GPS, and has an accuracy of ±3.1 cm or less. In a building or underground, tunnels or facilities may be affected. Data collection and area calculation through RTK can calculate the area using the AutoCAD program with the track record obtained by the surveyor wearing the surveying device and walking along the border of the year, just like the GPS survey method. In addition, since GPS and RTK-GPS surveys use tracks secured while moving, not stationary surveys, in order to reduce errors caused by differences in the investigator's stride length and physical strength, one surveyor equips the two equipments together and moves the boundary. You can record tracks.

<Survey using orthoimage>

DJI's MAVIC 2 PRO can be used as an orthographic device. This device is equipped with a 1-inch CMOS sensor with 20 million pixels. The flight altitude can be established to fly at 150 m using the DEM value of the National Geospatial Information Service. Using the orthoimage created through this, it can be displayed to determine the boundary where the event will occur with the naked eye.

<Image processing method>

Photos or videos taken by drones can be processed into aerial photographic (or video) data using Pix4D Mapper. Each identifiable image key point is automatically extracted from the captured photo, and then the key points extracted from each image are linked to each other through mutual comparison between aerial photos. Linked singularities are geometrically corrected by Automatic Aerial Triangulation (AAT) by reflecting ground control point information. The singular points of each geometrically corrected aerial photograph are automatically matched to the same points in the photograph taken through comparative analysis between successive aerial photographs. When matched singular points are extracted, point cloud data (3D Cloud Point) having 3D coordinates is created. A numerical surface model can be created from this point cloud data, and an orthoimage can be produced through the combination of aerial photographs.

Meanwhile, two data, that is, a video or photo received from the 360-degree VR camera 400 and a video or photo received from the drone 500, must be synchronized based on the time domain, for example, a blasting event. When occurs, while the drone 500 is in the A state in the sky, it can be determined that the video viewed by the 360-degree VR camera 400 or the damaged data in the photo is in the B state. It is important to match each data on the time axis because it is to measure and understand the impact of a blasting event on adjacent buildings or facilities before, during, and after construction. To this end, each data can be reproduced in a HTML5 container layout equipped with multimedia content having a time value, an HTML5 document associated with a preset time point in e-learning content through an HTML5 iFrame, and an integrated control player. At this time, the basic concept of iFrame is described as follows.

<iFrame>

Referring to (d) of FIG. 3, it is an example of a screen composition of HTML5 interactive content including an integrated player on a screen in which multimedia and iFrame areas are separately arranged. The content screen can be configured in various ways according to the content and purpose. It is common to place the content control player at the bottom of the screen in a horizontal format, but there is no set standard. Elements that make up content can be separated so that free screen composition is possible. At this time, the components can be classified into the above three types in consideration of the function, role, and file format of the content.

<Multimedia content with time value>

It is an element that leads the central content of the content to the center of the visual and auditory experience. Multimedia becomes a standard for the entire time value of content. It provides user interaction and content control experience by using a dedicated player based on <video> and <audio> tags of HTML5. In general, multimedia contents are produced using professional imaging equipment or smart devices capable of capturing images. The original video produced is edited using a separate program, and then encoded into MP4 and MP3 file formats, which are HTML5 standard formats, and applied. Production and editing of multimedia contents is a completely independent task from web publishing. Modifications that occur after multimedia content is loaded or linked to the HTML5 screen require the original video or sound source to be modified separately, and then the file itself is replaced.

<HTML5 document presented via iFrame>

Additional information associated with a specific point in time of multimedia within the content classifies the document by page unit of the displayed screen. The HTML5 document divided by unit page is created as a split file and is called as an iFrame according to a set time value. Since the divided HTML5 unit page documents have a relatively simple document structure and a small amount of writing, maintenance such as creation, modification, and deletion is easy. In addition, it is possible to implement various user manipulation functions by applying HTML5, CSS, and JS within individual pages, or to independently implement elegant performance using graphic animations. By dividing and configuring the unit pages in the iFrame, it is possible to increase the efficiency of development operation, such as differentially deploying web publishing technical personnel or dividing and developing by multiple personnel, considering the content planning intention and implementation level.

<HTML5 container layout with integrated control player>

The HTML5 container page, which is the basic layout of the content, includes the HTML5 player, which is a common feature of the content. It also plays a role in designating and merging the arrangement area and size of multimedia and iFrames. The multimedia time value can be changed through the player of the container page and the changed time value can be delivered to the container HTML5 document. The player is synchronized so that it can be called as an iFrame by linking HTML5 unit documents required at a specific point in time that is changed by user operation. The integrated control player of the container, which is a common function, can be modified and transformed according to the functional requirements. Since HTML5 player-related materials are being shared as open sources on the web, it is also possible to utilize them. However, content using the HTML5 content container structure is produced and operated in compliance with web standards, so it is distinguished from general video editing authoring tools, and unlike video platform services, it is possible to provide services independently on general websites.

<Convergence of multimedia and HTML5 documents>

Multimedia content having continuity in time and additional information content in a stationary state are all different in production and editing, content output and control methods. Therefore, an integrated control function that can fuse these two elements into one content is an essential component of interactive HTML5 multimedia convergence content. Players that support direct user manipulation in multimedia interactive content basically implement HTML5 tags, and functions such as multimedia playback, stop, timeline, subtitles, voice control, speed and section movement can be added and interlocked if necessary. there is. As described above, the multimedia playback time is a standard for all content time value information. Time value information that flows sequentially from the start of multimedia playback is periodically input to the integrated player. The entered information is transferred back to the HTML5 container. If the time value information previously input to the HTML5 container and the time value information generated from the multimedia are the same, the corresponding HTML5 unit page document is called through the iFrame.

When arbitrary time value information is generated by the user's manipulation of the player, the player transfers the changed time value information to the multimedia content and the HTML5 container. The time value delivered to the multimedia synchronizes the video to be reproduced from that point in time. The time value passed to the HTML5 container calls the HTML5 unit page in the same way as in sequential playback. By sharing the same time value information in a separated structure, it can be operated as one converged content by collectively controlling and synchronizing while maintaining the development characteristics and original form of contents of various formats.

The interlocking unit 350 matches the shooting time of the spherical photo or video received from the 360 degree VR camera 500 and the shooting time of the real-time drone data in units of frames, and in case of a photo, the shooting time and video In this case, frame intervals can be synchronized. Since this is the same as the above-mentioned iFrame, detailed information will be omitted.

The location identification marking unit 360 may mark a location where at least one event occurs in real-time drone data with a location icon, and mark and output a building adjacent to a location where at least one event occurs in real-time drone data.

When damage data is manually input from the user terminal 100 to the spherical photo or image received from the 360-degree VR camera 500, the automatic magnification unit 370 measures the area of a preset area centered on the coordinates where the input event occurred. It can be set to partially enlarge and display a photo or video of an input location.

The remote control unit 380 grants the user terminal 100 a remote control right to control the position of the drone 500, and the user terminal 100 that has obtained the remote control right controls the shooting direction, location, and altitude of the drone 500. In the case of control, a control event may be transmitted to the drone 500 in real time.

The comparison screen providing unit 390 may compare damage data and real-time drone data before (Pre) and after (Post) occurrence of at least one event and provide a comparison screen. This can also be compared using the aforementioned iFrame.

The artificial intelligence unit 391 may use at least one artificial intelligence algorithm to extract damage data including location, type, size, length, and shape of damage from a photo or image. Contents not described in detail in the present invention will refer to the prior registered patent of the present applicant as described above.

Hereinafter, an operation process according to the configuration of the survey service providing server of FIG. 2 described above will be described in detail with reference to FIGS. 3 and 4 as examples. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

Referring to FIG. 3 , (a) the survey service providing server 300 receives an image or photo from the 360 degree VR camera 400 and overlays damage data on the image or photo. In addition, (b) the survey service providing server 300 synchronizes the real-time drone data received from the drone 500 as shown in (c) and FIG. Or, photograph changes and damages of facilities, automatically perform artificial intelligence labeling, and display GPS in real time to determine which part is a facility, which part is a building, and which part is a year-end even in the video taken from the top. etc. to be able to understand. (d) The synchronization process may use iFrame, but is not limited thereto.

Matters that have not been explained about the method of providing roadside survey services using drones in FIGS. 2 to 4 are the same as those described for the method of providing roadside survey services using drones through FIG. 1 or from the described content. Since it can be easily inferred, the following description will be omitted.

5 is a diagram illustrating a process of transmitting and receiving data between each component included in the system for providing a roadside survey service using a drone of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process of transmitting and receiving data between each component will be described through FIG. 5, but the present application is not limited to such an embodiment, and according to various embodiments described above, It is obvious to those skilled in the art that a process of transmitting and receiving data may be changed.

Referring to FIG. 5, the investigation service providing server stitches the spherical photo or image received from the 360-degree VR camera to perform flattening (S5100), and the location and type of damage from the spherical photo or image , damage data including size, length and shape are extracted (S5200).

In addition, the survey service providing server transmits damage data to the user terminal (S5300), synchronizes the time between the damage data and the real-time drone data, and outputs the damage data and the real-time drone data (S5400).

The order between the above-described steps (S5100 to S5400) is only an example, and is not limited thereto. That is, the order of the above-described steps (S5100 to S5400) may be mutually changed, and some of the steps may be simultaneously executed or deleted.

Matters that have not been explained about the method of providing roadside survey services using drones in FIG. 5 are the same as those described for the method of providing roadside survey services using drones through FIGS. 1 to 4, or Since it can be easily inferred, the following description will be omitted.

The method of providing annual survey service using a drone according to an embodiment described with reference to FIG. 5 may be implemented in the form of a recording medium including instructions executable by a computer such as an application or program module executed by a computer. can Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described method for providing a yearly survey service using a drone according to an embodiment of the present invention is performed by an application basically installed in a terminal (this may include a program included in a platform or operating system basically installed in the terminal) It may be executed, or it may be executed by an application (ie, a program) directly installed in the master terminal by a user through an application providing server such as an application store server, an application or a web server related to the corresponding service. In this sense, the method for providing annual survey service using a drone according to an embodiment of the present invention described above is implemented as an application (i.e., a program) that is basically installed in a terminal or directly installed by a user, and is implemented as a computer such as a terminal. It can be recorded on a readable recording medium.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (7)

A drone that captures a place where at least one event requiring yearly survey occurs from the air, and performs real-time streaming to compare before and after the occurrence of the at least one event;
A 360-degree VR camera that captures 360 degrees horizontally and vertically to create spherical photos or videos;
Damage data, including the location, type, size, length, and shape of the damage extracted and labeled in the photo or video received from the 360-degree VR camera, is overlaid on the photo or video, and streamed from the drone in real time. A user terminal that synchronizes and outputs drone data with the damage data; and
A flattening unit that performs complanation by stitching spherical photos or images received from a 360-degree VR camera, and extracts damage data including location, type, size, length, and shape of damage from spherical photos or images. An extraction unit, a transmission unit that transmits damage data to a user terminal, a synchronization unit that synchronizes time between the damage data and the real-time drone data and outputs the damage data and the real-time drone data, and a spherical surface received from the 360-degree VR camera. When damage data is manually input from the user terminal to a photo or image, an automatic magnification unit configured to partially enlarge and display the photo or image of the input location in a preset area centered on the coordinates where the input event occurred; A remote control unit that transmits a control event to the drone in real time when a user terminal grants a remote control right to control the position of the drone, and the user terminal that has obtained the remote control right controls the shooting direction, location, and altitude of the drone. Including; a survey service providing server that includes;
The survey service providing server,
Interlocking unit that synchronizes the shooting time of the spherical photo or video received from the 360-degree VR camera and the shooting time of the real-time drone data in units of frames, and synchronizes the shooting time in case of a photo and the frame section in case of a video ;
a location identification marking unit for marking a location where the at least one event occurs in the real-time drone data with a location icon, and marking and outputting a building adjacent to the location where the at least one event occurs in the real-time drone data;
a comparison screen providing unit that provides a comparison screen by comparing damage data and real-time drone data before (Pre) and after (Post) the occurrence of the at least one event;
An artificial intelligence unit using at least one artificial intelligence algorithm to extract damage data including the location, type, size, length and shape of the damage from a photo or video; delete delete delete delete delete delete

Images