KR102243810B1 - Caption adding system and method of helicam recording image with position information for filming site - Google Patents

Abstract

The present invention is to provide a system for synthesizing location and attribute information through a caption by using a geographic information system when real-time photographing image information of a helicam is provided to an operator through a network. Accordingly, the present invention relates to a system for generating location and attribute information in a real-time photographing image of a helicam by a caption which comprises: a helicam that flies using variably rotational force of a rotor; a ground control device that receives location information, image information, a photographing angle, and photographing altitude information of the helicam in real time and transmits them to a streaming server and remotely controls and operates the helicam on the ground; and an image processing device for applying the location and attribute information to the image information of the helicam received through the streaming server by a caption.

Description

{Caption adding system and method of helicam recording image with position information for filming site}

The present invention relates to a method and system for generating position and attribute information as captions in a real-time photographed image of a helicam, and more particularly, utilizing a geographic information system when providing real-time photographed image information of a helicam to an operator through a network. The present invention relates to a method and system for synthesizing and outputting real-time location and attribute information of a target point, such as a current location and background, and a building.

Helicam is a remote radio-controlled photographing equipment made by attaching a camera to a small unmanned helicopter for shooting lively images or in places where people are difficult to access. It is possible to shoot at various angles while flying at high altitude, and to remotely control it. Because of this, it is possible to produce lively images and is useful in environments where human access is difficult, such as rock walls, fires, chemicals, and radioactive leakage accident sites.

As open source software for wireless control is recently developed and accessibility increases, the helicam is being used in various fields such as broadcasting shooting, 3D spatial information acquisition, communication, agriculture, and delivery.

However, since the existing helicam is a method that transmits the longitude and latitude acquired using the satellite navigation system (GPS) to the operator as the location information of the shooting location, it is very easy to accurately and quickly grasp the location information of the real-time image of the helicam. There is a difficult limit.

In addition, in order for the operator to determine the current location information of the helicam, it is very cumbersome, as it is very cumbersome because the operator has no choice but to check the information of the satellite navigation system (GPS) by accessing a portal site such as Google and converting it into an address. There is a problem that is difficult to cope with.

On the other hand, since the helicam is controlled wirelessly, it is vulnerable to hacking such as intercepting flight records and recorded images, hijacking by sending a GPS signal or malfunctioning signal, or rendering it inoperable.

In other words, the wireless communication between the helicam and the ground control device (GCS) is weak, so it is possible for a malicious attacker within the accessible range to attack by disguised as a legitimate ground control device.

For example, a helicam that does not have security applied does not verify the data transmitted through the wireless channel without verifying it, so if it receives an incorrect control command from an unjust ground control device (fault injection attacks), it is necessary to carry out it and maintain the flight. Malfunctions that unnecessarily consume power may occur, resulting in an inability to perform the original task and result in an inoperable state.

In addition, since cameras, sensors, and separate weapons for attack can be mounted, it is possible to incapacitate attacks that cause physical damage such as collisions and attacks of major facilities, as well as damages such as privacy invasion, personal information leakage, and video information leakage.

Therefore, there is a need for a technology that enhances safety through communication security between helicam and ground control system (GCS), control system, device authentication, safe operation of controllers, personal information protection, network security, and malicious code prevention.

The background technology or the prior art described herein is information possessed by the present inventor or acquired in the process of deriving the present invention, and is only intended to help understand the technical significance of the present invention, and the technology to which the present invention belongs prior to the filing of the present invention. It is stated that it does not mean a well-known technology in the field.

KR 10-2085051 B1(2020.02.28) KR 10-2178383 B1(2020.11.06) KR 10-2130973 B1(2020.07.01)

Accordingly, the present inventor comprehensively considers the above-described matters and at the same time provides a geographic information system when providing real-time photographed image information of the helicam to the operator through a network in the idea of solving the technical limitations and problems of the existing helicam technology. By utilizing the real-time location and property information of the current filming location by synthesizing and outputting it as a subtitle, the operator or manager who monitors it can accurately and quickly grasp the location and property information in the filmed image to quickly respond to emergency situations, etc. In addition, every effort to develop a new method and system for generating position and attribute information as subtitles in real-time images of the helicam that can further enhance the communication security between the helicam and the ground control system (GCS). As a result, the present invention was invented while continuing to study with inclination.

Therefore, the technical problem and object to be solved by the present invention is a method of generating position and attribute information as a caption on a real-time photographed image of a helicam so that the position and attribute information of the captured image of the helicam can be accurately and quickly identified, and It is in providing the system.

Another technical problem and object to be solved by the present invention is to provide a method and system for generating position and attribute information as captions in a real-time photographed image of a helicam to enhance security between a helicam and a ground control device. There is.

Here, the technical problems and objects to be solved by the present invention are not limited to the technical problems and objects mentioned above, and other technical problems and objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

A specific means according to an aspect of the present invention for effectively achieving a specific technical purpose while specifying a new concept for solving the technical problem of the present invention as described above is to fly by variably using the rotational force of the rotor. However, a GPS receiver that receives GPS information from a satellite, extracts NMEA (The National Marine Electronics Association) data and transmits it to the Virtual Reference Station (VRS) server of the National Geographic Information Institute, and VRS correction data from the VRS server of the National Geographic Information Institute. A positioning unit that receives and generates real-time location information, an image photographing unit that converts a subject into an image signal by real-time aerial photographing, a flight control control unit that controls the flight direction and posture and adjusts the photographing angle of the image photographing unit, and the image photographing A photographing angle measuring unit that measures a negative photographing angle, a photographing altitude measuring unit that measures the absolute altitude between the focus and the index of the image photographing unit, and an acquisition information transmitting unit that transmits the location information, image information, photographing angle and photographing altitude information in real time. A helicam configured to include, receiving location information, image information, shooting angle and shooting altitude information of the helicam from the acquisition information transmitting unit in real time and transmitting it to a streaming server, and remotely controlling and controlling the helicam from the ground. Ground control device, a reference coordinate identification unit that identifies the GPS coordinates of the ground reference point previously arranged in the image information area of the helicam received through the streaming server, the location information of the helicam, the shooting angle and the shooting altitude information, and the A coordinate estimating unit that estimates specific coordinates for a target point on the image information of the image capture unit using the GPS coordinates of the reference coordinate identification unit, and location and attribute information corresponding to the estimated coordinates of the coordinate estimating unit from the GIS server of the geographic information system. Generating a caption in which the extracted geographic information extracting unit and the location and attribute information extracted from the geographic information extracting unit are applied as captions to the image information of the helicam received through the streaming server A system for generating position and attribute information in a real-time photographed image of a helicam as a caption, characterized in that it includes an image processing device including a unit, is presented.

Accordingly, the present invention can synthesize and output real-time location and attribute information of a target point into subtitles using a geographic information system when providing real-time photographed image information of a helicam to an operator through a network. By accurately and quickly grasping the location and attribute information in the captured image, it is possible to quickly respond to an emergency situation or the like.

In addition, as a preferred aspect of the present invention, the helicam further includes a communication security module for transmitting and receiving information with the ground control device using an encryption system, and the communication security module includes a password A password change information unit that receives and stores information on the change cycle and change rule of the Helicam, an information management unit that manages the information on the helicam and existing password information, the password change cycle and change rule, and the heli based on the existing password information. By including a password change unit for automatically generating new password information for the cam and a password setting unit for setting new password information as the password of the helicam, communication security between the helicam and the ground control device may be enhanced.

A specific means according to another aspect of the present invention is: (a) the GPS receiver of the helicam receives GPS information from the satellite, extracts NMEA data, and transmits the NMEA data to the VRS server of the National Geographic Information Institute, (b) The step of generating real-time location information by receiving VRS correction data from the VRS server of the National Geographic Information Service by the location positioning unit of the helicam, (c) the image capture unit of the helicam photographs the subject in real time and converts it into an image signal. , The photographing angle measuring unit and the photographing altitude measuring unit of the helicam measure the photographing angle and photographing altitude of the image photographing unit, and transmitting real-time to the ground control device through the acquisition information transmission unit, (D) the ground control device acquires the Receiving the location information, image information, shooting angle, and shooting altitude information of the helicam from an information transmission unit in real time and transmitting it to a streaming server, (e) the reference coordinate identification unit of the image processing device received through the streaming server. Identifying the GPS coordinates of the ground reference point previously arranged in the image information area of the helicam, (f) the coordinate estimation unit of the image processing device receives the location information, the shooting angle, and the shooting altitude of the helicam through the streaming server. Estimating specific coordinates for a target point on the image information of the image capturing unit using information and GPS coordinates of the reference coordinate identification unit, (g) a geographic information extracting unit of the image processing unit corresponding to the estimated coordinates of the coordinate estimating unit Extracting the location and attribute information from the GIS server of the geographic information system, (H) the caption generation unit of the image processing apparatus receives the position and attribute information extracted from the geographic information extraction unit through the streaming server. A method of generating position and attribute information in a real-time photographed image of a helicam as a caption, characterized in that it comprises the step of coating image information of a cam with a caption.

Accordingly, the present invention uses a geographic information system to synthesize location and attribute information into real-time photographed image information of helicam as subtitles, and through this, an operator or manager can accurately and quickly determine the position and attribute information in the photographed image. By grasping it, it can be possible to quickly respond in case of an emergency situation.

In addition, as another preferred aspect of the present invention, in the step (f), the coordinate estimation unit estimates the coordinates of the target point on the image information of the image capturing unit and the ground pre-arranged at the closest position to the estimated coordinates. By comparing the GPS coordinates of the reference points and verifying the estimated coordinates, position and attribute information corresponding to the actual coordinates in the real-time photographed image information of the helicam can be more accurately synthesized and output as a caption.

In addition, as another preferred aspect of the present invention, in the step (f), when the location information, the shooting angle, and the shooting altitude information of the helicam are changed, the helicam is Calculate by changing the estimated coordinates for the target point located at the center of the image information transmitted from the cam, and if the estimated coordinates are continuously changed and calculated, the distance between the previous estimated coordinates and the current estimated coordinates is calculated, and then the previous estimated coordinates The estimated coordinates are verified by comparing the distance between the GPS coordinate information of the ground reference point placed and stored in advance and stored in the nearest place and the GPS coordinate information of the ground reference point previously placed and stored in the nearest place to verify the real-time photographed image of the helicam. Position and attribute information corresponding to the actual coordinates in the information can be calculated more precisely.

According to the technical idea and embodiment of the present invention, on which a unique solution is based to solve the above technical problem, the location and attribute information of the target point is recorded in the real-time photographed image of the helicam using a geographic information system. By synthesizing and outputting the subtitles, the operator or manager who monitors them can accurately and quickly grasp the location and attribute information in the captured image, and can quickly respond and take measures in the event of an emergency situation, thereby minimizing damage.

In addition, by using VRS data, the accuracy of the shooting position of the helicam is improved, as well as by detecting the GPS coordinates of the ground reference point, and verifying the mutual distance and position with the estimated coordinates, it is possible to obtain accurate location and attribute information in the captured image. .

In addition, the security can be further strengthened by periodically automatically changing the encryption system for the connection of the communication system between the helicam and the ground control device.

Here, the effects of the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram schematically illustrating a system for generating position and property information as captions in a real-time captured image of a helicam according to an exemplary embodiment of the present invention.
2 is a block diagram schematically illustrating a communication security module constituting a system for generating position and attribute information as captions in a real-time photographed image of a helicam according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a display and verification process of a system for generating position and attribute information as captions in a real-time photographed image of a helicam according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

Prior to this, the terms to be described later are defined in consideration of functions in the present invention, and this should be interpreted as a concept conforming to the technical idea of the present invention and a meaning commonly or commonly recognized in the art.

In addition, when it is determined that a detailed description of a known function or configuration related to the present invention may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Here, the accompanying drawings are partially exaggerated or simplified for description of the configuration and operation of the technology, and for convenience and clarity of understanding, and it is understood that each component does not exactly match the actual size and shape.

In addition, in the present specification, the term and/or is meant to include a combination of a plurality of related items or any item among a plurality of related items, and when a certain part includes a certain element, it is a particularly opposite description. Unless there is no other component is not excluded, it means that other components can be further included.

In other words, terms such as'include','equipment', and'have' mean the presence of features, numbers, steps, actions, components, parts, or a combination of the features set out in the present specification. It is to be understood that it does not exclude the possibility of the presence or addition of one or more other features or numbers, step-operating components, parts, or combinations thereof.

In addition, each step may occur differently from the specified order unless a specific order is clearly stated in the context. That is, each of the steps may occur in the same order as the specified order, may be performed substantially simultaneously, or may be performed in the reverse order.

In addition, the meaning of the terms "unit" and "unit" refers to a unit or a module that serves to process at least one function or a certain operation desired in the system, which is hardware or software, or a combination of hardware and software. It may be implemented as a device or assembly capable of performing an independent operation or means through the like.

Also, terms such as first and second may be used to describe various elements. That is, terms such as first and second may be used only for the purpose of distinguishing one component from another component. For example, a first component may be referred to as a second component without departing from the protection scope of the present invention, and a second component may also be referred to as a first component.

In addition, the terms "transmission", "communication", "transmission", "receiving" and other similar meanings of signals or information are not only direct transmission of signals or information from one component to another component, but also other components. It includes what is transmitted through. In particular, "transmitting" or "transmitting" a signal or information as a component indicates the final destination of the signal or information and does not mean a direct destination. The same is true for "reception" of signals or information.

As shown in FIGS. 1 and 2, the main components constituting the system for generating position and property information as captions in the real-time photographed image of the helicam according to the embodiment of the present invention are the helicam 100 and the VRS server ( 200), a ground control device 300, a streaming server 400, an image processing device 500, and a GIS server 600, and the communication method between the object and the object is It can include any communication method that can be networked.

For example, the wired and/or wireless network between the helicam 100, the VRS server 200, the ground control device 300, the streaming server 400, the image processing device 500 and the GIS server 600 LAN(Local Area Network), MAN(Metropolitan Area Network), GSM(Global System for Mobile Network), EDGE(Enhanced Data GSM Environment), HSDPA(High Speed Downlink Packet Access), W-CDMA(Wideband Code Division Multiple Access) , Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Zigbee, Wi-Fi, Voice over Internet Protocol (VoIP), LTE Advanced, IEEE80216m, WirelessMAN -Advanced, HSPA+, 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE 80216e), UMB (formerly EV-DO Rev C), Flash-OFDM, iBurst and MBWA (IEEE 80220) systems, HIPERMAN, Beam-Division Multiple Access It may refer to a communication network using one or more communication methods selected from the group consisting of (BDMA), World Interoperability for Microwave Access (Wi-MAX), and ultrasonic communication, but is not limited thereto.

The helicam 100 variably uses the rotational force of the rotor to take a real-time aerial photograph of a subject while flying, converts it into an image signal, generates data such as location information, and transmits it in real time.

Specifically, the helicam 100 includes a GPS receiver 110, a positioning unit 120, an image capturing unit 130, a flight control control unit 140, a photographing angle measuring unit 150, and a photographing altitude measuring unit 160. , Acquisition information transmission unit 170 and a communication security module 180.

The GPS receiver 110 receives GPS information from an artificial satellite, extracts NMEA data, and transmits it to the VRS server 200 of the National Geographic Information Institute.

Here, NMEA (The National Marine Electronics Association) data refers to the signal format used in GPS, and data such as Global Positioning System Fix Data (GPGGA), GPS Satellites in View (GPGSV), and Recommended Minimum Specific GNSS Data (GPRMC) are available. This is all-inclusive raw data.

In addition, if the entire NMEA data is transmitted to the VRS server 200 of the National Geographic Information Institute in the form of a packet, the data volume is large and the transmission may not be performed smoothly because a lot of unnecessary information is included, so the operation of extracting GPGGA data from the NMEA data After passing through, it is preferable to transmit it to the VRS server 200. To this end, it goes without saying that the helicam 100 must include a data processing module.

Meanwhile, the GPS receiver 110 may be configured as a receiver of two or more frequencies supporting a network-based RTK method.

The location positioning unit 120 receives VRS correction data from the VRS server 200 of the National Geographic Information Institute and generates real-time location information of the helicam 100.

That is, the location positioning unit 120 may obtain accurate current location information of the helicam 100 through the VRS server 200.

The image capturing unit 130 photographs a subject in real time and converts it into an image signal.

That is, the image capturing unit 130 is mounted in the center of the body of the helicam 100 to detect an object and acquire an image by scanning the moving direction and the downward direction of the helicam 100.

Here, if the image capturing unit 130 is a photographing means capable of obtaining an image capable of distinguishing the shape of a subject to be photographed, such as a visible ray camera, an infrared camera, or a stereo camera, the type is not particularly limited.

In addition, by using a separate gimbal or a 3-axis motion stabilizer system for camera, the image capture unit 130 minimizes image shaking caused by external actions such as wind during flight, thereby providing a stable image. Can be acquired.

Meanwhile, the image capturing unit 130 may perform efficient signal processing by compressing and transmitting image information.

The flight control control unit 140 is composed of a control processor and controls the flight direction, posture, speed, and altitude of the helicam 100 and adjusts the shooting angle and shooting conditions of the image capturing unit 130.

Here, the flight control control unit 140 may be implemented as separate hardware or may be implemented in the same hardware.

For example, it can be implemented on a processor such as the same ASIC, CPU, or FPGA, or can be implemented on a separate processor.

In addition, the flight control control unit 140 may adjust the photographing angle by tilting the image capturing unit 130 to the left or right according to the manipulator's manipulation, or rotate it 360 degrees around the vertical reference axis.

For example, it is possible to control the degree of rotation and tilt of the image capturing unit 130 by adjusting the number of rotations using a motor.

In addition, the flight control control unit 140 may control to maintain the photographing posture by adjusting the roll, pitch, and yaw angle of the image capturing unit 130 using a gimbal and a navigation coordinate system.

The photographing angle measuring unit 150 measures the photographing angle of the image photographing unit 130.

That is, the photographing angle measurement unit 150 calculates the inclined angle of the image capturing unit 130 with respect to the helicam 100 around a reference axis in the vertical direction, and calculates an angle rotated clockwise from the reference axis. .

The photographing altitude measurement unit 160 measures the photographing altitude of the image photographing unit 130.

That is, the photographing altitude measuring unit 160 calculates the photographing altitude of the image photographing unit 130 by measuring the absolute altitude between the camera focus of the image photographing unit 130 and the index.

The acquisition information transmission unit 170 transmits location information, image information, photographing angle, and photographing altitude information based on a wireless network in real time according to a wireless communication protocol.

The communication security module 180 performs communication with the ground control device 300 using an encryption system.

Specifically, the communication security module 180 prevents data leakage during the communication process and performs encrypted communication for authentication to the network, and the password change information unit 181, the information management unit 182, the password change unit 183, and the password It includes a setting unit 184.

The password change information unit 181 receives and stores information on a change period and change rule of a password.

For example, the password change period can be automatically generated based on a change rule consisting of a combination of alphabetic characters, numbers, and special characters when the maximum period of use is set, and when the period of use has elapsed.

The information management unit 182 manages information on the communication system of the helicam 100 and existing password information.

Here, the existing password information may be password information for any one of a personal account, a shared account, and an application account for communication with the communication system of the helicam 100.

The password change unit 183 automatically generates new password information for communication with the communication system of the helicam 100 based on the password change cycle and change rule, and existing password information.

In addition, the password changer 183 may automatically generate new password information based on at least one of a password change period and an administrator control.

In this case, the new password information may be information of a password for any one of a personal account, a shared account, and an application account of the communication system of the helicam 100 corresponding to the existing password information.

In addition, the password change unit 183 may change the password of the communication system of the helicam 100 in an automatic change or request change method. The automatic change method is a method of changing the password of the communication system through periodic automatic change, random change, and rule change according to the password change period, and the request change method is a method of changing the password of the communication system through user control, that is, approval processing. to be.

The password setting unit 184 sets the new password information generated by the password changing unit 183 as the password of the communication system of the helicam 100.

In this case, the password setting unit 184 may change the password of the application account of the communication system of the helicam 100 into new password information by using a push method or a polling method.

In addition, when the password of the application account of the communication system of the helicam 100 is present in the script or environment file, the password setting unit 184 stores the new password information in the communication system of the helicam 100 and the application and script or environment. You can change the password information in a format that synchronizes to a file.

Therefore, it is possible to further enhance security by periodically changing the encryption system for accessing the communication system between the helicam 100 and the ground control device 300.

On the other hand, the fuselage of the helicam 100 includes a GPS device that allows you to control the aircraft with wireless radio waves, a sensor necessary for safe flight, a communication system for wireless communication with the ground control device 300, and a power including a rotorcraft and a battery. Modules and software modules that control them are mounted.

For example, inside the body of the helicam 100, a software module is installed that is electrically connected to a power module and controls its driving and driving directions according to whether current is applied and the applied direction.

In addition, the helicam 100 may be equipped with autonomous flight software (FMS) for flying along a flight path based on 3D coordinate information.

Here, the aircraft of the helicam 100 is not only a rotary wing drone, but also a fixed wing drone, a tilt-rotor drone, and a drone using an impeller system that obtains lift and propulsion through air intake and discharge. Not limited.

In particular, in the case of a rotorcraft drone, the flight module may include a propeller (prop) that is rotated by the drive of a motor, and in this case, the flight control in a rotorcraft drone is controlled by the propeller (rotation speed of the propeller or variable pitch of the propeller, etc.) Flight attitude, path, and take-off and landing can be controlled through a fixed-wing drone or impeller system drone, etc., the wing and the flight module may include an air intake or discharge system, a propeller, etc., and flight control can include the angle and rotation of each part. It is possible to control parts or structures for controlling propulsion and lift according to the shape and structure of the drone, such as controlling water and motor driving.

Meanwhile, each unit or module included in the helicam 100 may be entirely hardware or partially hardware and partially software. For example, it may collectively refer to hardware for processing data of a specific format and content or/or sending and receiving data in an electronic communication method, and software related thereto.

In addition, in the present specification, terms such as "unit", "module", "device", "terminal", "server" or "system" may refer to a combination of hardware and software driven by the hardware.

For example, the hardware may be a data processing device including a CPU or other processor.

In addition, software driven by hardware may refer to an executing process, an object, an executable file, a thread of execution, a program, and the like.

In addition, each component constituting the helicam 100 is not necessarily intended to refer to separate devices that are physically separated from each other.

That is, the positioning unit 120, the shooting angle measurement unit 150, the shooting altitude measurement unit 160, the acquisition information transmission unit 170 and the communication security module 180, etc. It is functionally classified according to an operation performed by the corresponding hardware, and each unit is not necessarily provided independently of each other. Of course, depending on the embodiment, one or more of the respective components may be implemented as separate devices that are physically separated from each other.

The VRS server 200 receives NMEA data from the GPS receiver 110 of the helicam 100, generates VRS correction data, and transmits it to the positioning unit 120 of the helicam 100 through a network.

That is, the VRS server 200 receives NMEA data or GPGGA data extracted therefrom from the helicam 100 to generate VRS (Virtual Reference Station) data of the current location.

Here, the VRS correction data is used for position correction of the NMEA data extracted from the GPS receiver 110 to increase the accuracy of the current coordinates of the helicam 100.

That is, the VRS server 200 separates and models the error using a reference station network consisting of GPS stations, creates a virtual reference station at a point near the GPS receiver 110, and then calculates using the network formed in the actual reference station. After receiving the corrected value and recalculating the position of the virtual reference station, the correct position of the GPS receiver 110 may be determined and transmitted to the position positioning unit 120 of the helicam 100.

The ground control device 300 remotely controls and controls the helicam 100 from the ground, and receives the location information, image information, shooting angle and shooting altitude information from the helicam 100 in real time, and the streaming server 400 It is transmitted to the image processing apparatus 500 through.

Here, the ground control device 300 is a control module through which a plurality of helicams 100 can be simultaneously connected and controlled in a multi-drop method or a pilot can directly control the helicam 100, and the flight route of the helicam 100 It may be configured to include a mission module, etc. that can be set in advance, and may include a joystick for manual manipulation of a manipulator, and a signal generator that generates a control signal according to the manual manipulation of the joystick.

In addition, before actual communication with the helicam 100 is performed, it is preferable that a pairing process for connecting to each other is preceded, and in this process, the unique number of the helicam 100 is received and stored and managed. It is advisable to share it as encrypted data.

The streaming server 400 provides the location information, image information, shooting angle and shooting altitude information of the helicam 100 transmitted from the acquisition information transmission unit 170 of the helicam 100 in real time through the ground control device 300. It receives and collects and transmits it to the image processing apparatus 500.

Here, the streaming server 400 may transmit information collected by accessing a mobile communication base station through a wireless communication network such as Wi-Fi, 4G, 5G, etc. and a terminal to the image processing apparatus 500 through real-time streaming.

The image processing device 500 uses image information, location information, shooting angle information, shooting altitude information, and GPS coordinates of a pre-arranged and stored ground reference point of the helicam 100 received through the streaming server 400. Specific coordinates for the target point on the information are estimated, location and attribute information corresponding to the estimated coordinates are extracted from the GIS server 600 of the geographic information system, and the corresponding image information is covered with captions and stored.

Specifically, the image processing apparatus 500 includes a reference coordinate identification unit 510, a coordinate estimation unit 520, a geographic information extraction unit 530, a caption generation unit 540, and an integrated information receiving unit 550.

The reference coordinate identification unit 510 detects and identifies GPS coordinates of a ground reference point previously arranged and stored in the image information area of the helicam 100 received through the streaming server 400.

That is, the reference coordinate identification unit 510 is configured to pre-arrange a plurality of images in the image information area of the helicam 100 and to store the GPS coordinate information and the location of the ground reference point in advance, and received by the integrated information receiving unit 550 It is configured to identify the image of the GPS coordinates of the ground reference point arranged in advance from the image information of the helicam 100.

Here, the method of identifying the image of the GPS coordinates of the ground reference point may be performed by a method of extracting from the image pixels in which a shape corresponding to the GPS coordinates of each ground reference point and a unique color array are located, or an image processing algorithm.

In addition, the reference coordinate identification unit 510 may be configured to store the image information of the helicam 100 received by the integrated information receiving unit 550, and the image information according to the shooting altitude of the helicam 100 and It is also possible to store accumulation information indicating the percentage of the actual image.

The coordinate estimation unit 520 uses the flight direction, flight speed, location information, photographing angle and photographing altitude information of the helicam 100 and the image information of the helicam 100 using the GPS coordinates of the reference coordinate identification unit 510 Estimate specific coordinates for the target point on the image.

That is, the coordinate estimating unit 520 uses the current location information and flight status information of the helicam 100, the shooting angle and shooting altitude information of the image capturing unit 130, and the angle of view information stored in advance of the image capturing unit 130. It can be calculated by estimating specific coordinates for a target point existing in the space on the image information or located in the center by triangulation.

For example, the coordinate estimating unit 520 subtracts the altitude value of 3D coordinate information representing the flying terrain from the current location information and altitude value of the helicam 100, and the length of the repair line, that is, at the center of the image information. You can calculate the distance to the target point where it is located.

Here, since the image information of the helicam 100 is photographed at one fixed point in a wide area, there may be distortion of the image due to spherical aberration of the lens installed in the image capturing unit 130. The coordinate estimation unit 520 compares the coordinates estimated for the target point on the image information of the image taking unit 130 and the GPS coordinates of the ground reference point previously arranged and stored in the closest position to the estimated coordinates to be estimated coordinates. To verify.

For example, the image information of the image capturing unit 130 is an image having a resolution of 200×100 according to the number of pixels in the horizontal and vertical directions, and the center coordinate of the image is called (0, 0), and the identified ground Assuming that the GPS coordinates of the ground reference point located closest to the center of the image among the GPS coordinates of the reference point are (5, -8), the GPS coordinates of the ground reference point located closest to the center of the image (5, -8) and Since the actual distance between estimated coordinates adjacent thereto may be calculated through pre-stored GPS coordinates, the actual distance between the GPS coordinates of the ground reference point and the estimated coordinates adjacent thereto can be projected onto the image to verify the position in the image information.

In this case, the ratio of the distance on the actual ground surface and the distance in the image is determined by using the photographing altitude information of the helicam 100 and the photo scale information of the image photographing unit 130 stored in the reference coordinate identification unit 510 when photographing the image information. It can also be calculated.

Meanwhile, the method of verifying the estimated coordinates by the coordinate estimating unit 520 may be implemented in the form of a program command that executes a command through various means for electronically processing information and recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like alone or in combination.

The geographic information extracting unit 530 extracts location and attribute information corresponding to the estimated coordinates of the coordinate estimating unit 520 from the GIS server 600 of the geographic information system.

That is, the geographic information extraction unit 530 connects to the GIS server 600 of the geographic information system and extracts location and attribute information corresponding to the estimated coordinates of the coordinate estimation unit 520.

The caption generating unit 540 coats the location and attribute information extracted from the geographic information extracting unit 530 onto the image information of the helicam 100 received through the streaming server 400 as captions.

That is, the caption generation unit 540 analyzes the location and attribute information extracted from the geographic information extraction unit 530, and the target point located at the center of the image information of the helicam 100 received through the streaming server 400 If there is location and attribute information matching with, it may be detected as text or number matching with the corresponding image information, and the corresponding image information may be captioned with location and attribute information such as coordinates, addresses, names of nearby buildings, or major terrain names.

For example, when the modeling of an area processed as a caption in a virtual space with 3D coordinates or a dext or numeric object having attributes is referred to as a caption object, the helicam 100 photographed in a real space. The screen of the virtual space with the caption object viewed from the point of view of the actual image information may be merged with the actual image information and then combined, and then output to a monitoring screen or stored in a storage device.

The integrated information receiving unit 550 receives and outputs the location information, image information, photographing angle, and photographing altitude information of the helicam 100 through the streaming server 400.

That is, the integrated information receiving unit 550 may output the integrated information received through the streaming server 400 to a separate display terminal or store it in a storage device.

The GIS server 600 provides map-based geographic information, that is, spatially distributed information, in a database.

Here, the database of the GIS server 600 consists of spatial data and attribute data, and the spatial data are the type, location, size, and spatial topological relationship with other terrain elements, and the attribute data is the attribute of the terrain element. There are quantitative data including the mathematical meaning of Korea and qualitative data necessary for the description of the object such as map name, cycle, and label.

A method of generating position and attribute information as captions in a system that generates position and attribute information as captions in a real-time captured image of a helicam according to an embodiment of the present invention configured as described above will be described with reference to FIGS. 1 and 2. Is the same as

First, the helicam 100, while flying according to the control and control of the ground control device 300, the GPS receiver 110 receives GPS information from an artificial satellite, extracts NMEA data, and the VRS server 200 of the National Geographic Information Institute. Transfer to.

In addition, the location positioning unit 120 of the helicam 100 receives VRS correction data from the VRS server 200 of the National Geographic Information Institute and generates real-time location information.

Subsequently, the image capturing unit 130 of the helicam 100 photographs the subject in real time and converts it into an image signal, and the shooting angle measurement unit 150 and the shooting altitude measurement unit 160 of the helicam 100 The photographing angle and photographing altitude of the image capturing unit 130 are measured and transmitted to the ground control device 300 in real time through the acquisition information transmitting unit 170.

The ground control device 300 receives the location information, image information, shooting angle, and shooting altitude information of the helicam 100 from the acquisition information transmission unit 170 in real time, and the image processing device 500 through the streaming server 400 Transfer to.

At this time, the reference coordinate identification unit 510 of the image processing apparatus 500 is placed and stored in the image information area obtained by the image capture unit 130 of the helicam 100 received through the streaming server 400 in advance. The GPS coordinates of the reference point are detected and identified.

Subsequently, the coordinate estimation unit 520 of the image processing apparatus 500 includes location information, photographing angle and photographing altitude information of the helicam 100 received through the streaming server 400, and the GPS of the reference coordinate identification unit 510. Using the coordinate information, specific coordinates for the target point on the image information acquired by the image capturing unit 130 transmitted through the streaming server 400 are estimated.

At this time, the coordinate estimating unit 520 is at a target point (newin building) located at the center (central point) of the image information acquired by the image capturing unit 130 transmitted through the streaming server 400 as shown in FIG. 3. The estimated coordinates of the estimated coordinates and the GPS coordinates of the pre-arranged and stored ground reference points among the GPS coordinates information of the pre-arranged and stored ground reference points are designated and selected by triangulation, and then the estimated coordinates are verified I can.

In addition, when the location information, the shooting angle, and the shooting altitude information of the helicam 100 are changed, the image capture unit 130 obtains and estimates a target point located at the center of the image information transmitted through the streaming server 400. The coordinates are changed and calculated using the GPS coordinates of the reference coordinate identification unit 510, and when the estimated coordinates are continuously changed and calculated, the distance between the previous estimated coordinates and the current estimated coordinates is calculated, and then the distance between the previous estimated coordinates and the current estimated coordinates is calculated. The estimated coordinates may be verified by comparing the distance between the GPS coordinate information of the ground reference point previously placed and stored at the location and the GPS coordinate information of the ground reference point previously placed and stored at the nearest location to the current estimated coordinates.

Subsequently, the geographic information extracting unit 530 of the image processing apparatus 500 extracts location and attribute information corresponding to the estimated coordinates of the coordinate estimating unit 520 from the GIS server 600 of the geographic information system.

Thereafter, the caption generation unit 540 of the image processing apparatus 500 acquires the location and attribute information extracted from the geographic information extraction unit 530 from the image capturing unit 130 and transmits the image through the streaming server 400. After captioning the information and synthesizing it, it can be output to a monitor or stored in a separate storage device.

As described above, the system for generating position and attribute information as captions in the real-time photographed image of the helicam according to an embodiment of the present invention estimates the coordinates of the target point on the real-time photographed image of the helicam 100, and the corresponding position And since the attribute information is synthesized and outputted as a caption, it is possible to more accurately and quickly grasp the location and attribute information of the target point in the captured image.

In addition, since real-time location information is generated by using VRS correction data received from the VRS server 200 of the National Geographic Information Institute, the accuracy of the shooting location of the helicam 100 is improved as well as the GPS coordinates of the ground reference point are detected and estimated. Since the mutual distance and position with the coordinates are verified, accurate position and attribute information in the captured image can be obtained.

The operation by the method of generating position and property information as captions in the real-time photographed image of the helicam according to the embodiment of the present invention described above may be implemented at least partially as a computer program and recorded on a computer-readable recording medium. In addition, the recording medium in which a program for implementing the operation by the method is recorded and the computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer is stored.

Examples of computer-readable recording media include ROM, RAM, CDROM, magnetic tapes, floppy disks, and optical data storage devices.

In addition, the computer-readable recording medium may be distributed over a computer system connected through a network, and computer-readable codes may be stored and executed in a distributed manner.

On the other hand, the present invention is not limited by the above-described embodiment and the accompanying drawings, and can be variously modified and applied in various ways not illustrated within the scope not departing from the technical spirit of the present invention. It will be apparent to those of ordinary skill in the art that the substitution of components and the possibility of wide application by changing to other equivalent embodiments are possible.

Therefore, the contents related to modification and application of the technical features of the present invention should be interpreted as being included within the spirit and scope of the present invention.

100: helicam
200: VRS server
300: ground control device
400: streaming server
500: image processing device
600: GIS server

Claims (4)

A GPS receiver 110 installed on the helicam 100 that variably uses the rotational force of the rotor to fly, and extracts NMEA data by receiving GPS information from an artificial satellite and transmits it to the VRS server 200 of the National Geographic Information Institute;
A location positioning unit 120 installed on the helicam 100 and receiving VRS correction data from the VRS server 200 of the National Geographic Information Institute to generate real-time location information;
An image photographing unit 130 installed on the helicam 100 and converting a subject into an image signal by photographing a subject in real time;
A flight control control unit 140 for controlling a flight of the helicam 100 and a photographing angle of the image capturing unit 130;
A photographing angle measurement unit 150 for measuring a photographing angle of the image photographing unit 130;
A photographing altitude measuring unit 160 installed on the helicam 100 and measuring an absolute altitude between the focus of the image capturing unit 130 and an index;
An acquisition information transmission unit 170 for acquiring location information, image information, shooting angle, and shooting altitude information of the helicam 100 and transmitting it in real time;
A password change information unit 181 installed in the helicam 100 and performing communication with the ground control device 300 using an encryption system, but receiving and storing information on a password change cycle and change rule, and the Information on the communication system of the helicam 100 and the communication system of the helicam 100 based on the information management unit 182 that manages the information on the communication system of the helicam 100 and the existing password information, the password change cycle and change rules, and the existing password information. A communication security module 180 comprising a password changing unit 183 for automatically generating new password information and a password setting unit 184 for setting new password information as a communication system password of the helicam 100;
The location information, image information, photographing angle and photographing altitude information of the helicam 100 are received from the acquisition information transmitting unit 170 in real time and transmitted to the streaming server 400, and the helicam 100 is transmitted from the ground. A ground control device 300 for remote control and control;
The heli received through the streaming server 400 and installed in the image processing device 500 that analyzes and processes necessary information based on the image information of the helicam 100 received through the streaming server 400 A reference coordinate identification unit 510 for detecting and identifying GPS coordinates of a ground reference point previously arranged and stored in the image information area of the cam 100;
The image of the image capture unit 130 is installed in the image processing device 500 and uses the location information, the shooting angle, and the shooting altitude information of the helicam 100 and the GPS coordinates of the reference coordinate identification unit 510. A coordinate estimating unit 520 for estimating specific coordinates for a target point on the information;
A geographic information extracting unit 530 installed in the image processing apparatus 500 and extracting location and attribute information corresponding to the estimated coordinates of the coordinate estimating unit 520 from the GIS server 600 of the geographic information system;
A caption generating unit 540 for captioning the location and attribute information extracted from the geographic information extracting unit 530 on the image information of the helicam 100 received through the streaming server 400; And
An integrated information receiving unit 550 installed in the image processing apparatus 500 and receiving and outputting location information, image information, photographing angle and photographing altitude information of the helicam 100 through the streaming server 400;
It is made including,
The GPS receiver 110 of the helicam 100 receives GPS information from an artificial satellite, extracts NMEA data, and transmits it to the VRS server 200 of the National Geographic Information Institute,
The location positioning unit 120 of the helicam 100 receives VRS correction data from the VRS server 200 of the National Geographic Information Institute to generate real-time location information,
The image capturing unit 130 of the helicam 100 photographs a subject in real time and converts it into an image signal,
The photographing angle measurement unit 150 and the photographing height measurement unit 160 of the helicam 100 measure the photographing angle and photographing height of the image photographing unit 130, and control the ground through the acquisition information transmission unit 170. Real-time transmission to the device 300,
The ground control device 300 receives the location information, image information, shooting angle and shooting altitude information of the helicam 100 from the acquisition information transmission unit 170 in real time, and processes the image through the streaming server 400 Transfer to the device 500,
The reference coordinate identification unit 510 of the image processing device 500 detects and identifies the GPS coordinates of the ground reference point previously arranged and stored in the image information area of the helicam 100 received through the streaming server 400 and,
The coordinate estimation unit 520 of the image processing device 500 receives the location information, the shooting angle and the shooting altitude information of the helicam 100 received through the streaming server 400, and the reference coordinate identification unit 510 Estimating specific coordinates for a target point on the image information acquired by the image capturing unit 130 using GPS coordinate information of,
The coordinate estimation unit 520 of the image processing device 500 estimates the coordinates of the target point located at the center of the image information transmitted from the helicam 100, and pre-arranges the coordinates closest to the estimated coordinates, and The estimated coordinates are verified by comparing the stored GPS coordinate information of the ground reference point, but when the location information, the shooting angle, and the shooting altitude information of the helicam 100 are changed, the GPS coordinate information of the reference coordinate identification unit 510 If the estimated coordinates for the target point located at the center of the image information transmitted from the helicam 100 are changed and calculated, and the estimated coordinates are continuously changed and calculated, the distance between the previous estimated coordinates and the current estimated coordinates After calculating the estimated coordinates, the estimated coordinates are verified by comparing the distance between the GPS coordinate information of the ground reference point previously placed and stored in the nearest place to the previous estimated coordinates and the GPS coordinate information of the ground reference point previously placed and stored in the closest place to the current estimated coordinates and,
The geographic information extracting unit 530 of the image processing apparatus 500 extracts location and attribute information corresponding to the estimated coordinates of the coordinate estimating unit 520 from the GIS server 600 of the geographic information system,
The caption generation unit 540 of the image processing device 500 receives the location and attribute information extracted from the geographic information extraction unit 530 through the streaming server 400 to take an image of the helicam 100 A system for generating position and attribute information as a caption on a real-time photographed image of a helicam, characterized in that the caption is applied on the image information obtained by the unit 130. delete delete delete

Images