KR19990001299A - Operation method of UAV system - Google Patents

Abstract

The present invention relates to a method for operating an unmanned reconnaissance system, and more particularly, to a method for operating an unmanned reconnaissance system using augmented reality technology that is a technology for reinforcing real-world information using virtual reality.

The operation of the unmanned reconnaissance system uses a computer graphics technology and visualization method to visualize the reconnaissance target, collect information about the reconnaissance target, and generate a virtual image to obtain a realistic image of the reconnaissance target. A first process, a second process of synthesizing the virtual image and the real image generated in the first process by acquiring the absolute position and the actual image of the object of reconnaissance using a GPS and a CCD camera; and the unmanned reconnaissance system through a user interface It consists of a third process of synchronizing the actual image and the virtual image that changes according to the movement of the, can efficiently use the information, it is possible to increase the use of information by facilitating information acquisition.

Description

Operation method of UAV system

The present invention relates to a method for operating an unmanned reconnaissance system, and more particularly, to a method for operating an unmanned reconnaissance system using augmented reality technology that is a technology for reinforcing real-world information using virtual reality.

The unmanned reconnaissance system is a system for performing a reconnaissance work by a person in the past, the development of such a system was possible due to the development of wireless communication technology and computer graphics technology.

In addition, as the interest in the interaction between humans and computers is increasing in recent years, the importance of augmented reality is gradually increasing, and computer graphics technology, computer vision technology, image processing technology, wireless communication technology, and GIS, which are elements of augmented reality (Geometric Information System) data processing technology, Global Positioning System (GPS) operation technology and related technologies for applications are being integrated.

In this case, Augmented Reality technology is a recently introduced technology aimed at improving understanding of the real world. For example, the name of a part that is seen when looking into the inside of a car to a person who has no knowledge about the internal structure of the car and By providing information such as text, picture, and voice at the same time, it is a technology to improve understanding of real objects.

By developing a technology that integrates the research results of the related fields, a technique of reinforcing real-world information using virtual reality, a method of registering a virtual object in the real world, a method of expressing a virtual object in the real world, an object of the real world and a virtual world Research and development on element technologies of augmented reality, such as accurate measurement technology, real-time processing technology and user interface technology, has begun.

However, until now, there is a drawback that the consideration of the underlying technology of the human-oriented user interface is insufficient.

Therefore, the present invention is to provide a method of operating an unmanned reconnaissance system using augmented reality technology that can obtain a user-centered interface and real-time information.

In order to achieve the above object, the method of operating the unmanned reconnaissance system provided by the present invention collects information on a reconnaissance target by photographing the reconnaissance target with a CCD (Computer-Controlled Display) camera to obtain a realistic image of the reconnaissance target, The first step of visualizing necessary information using computer graphics technology and visualization method to generate a virtual image, and the absolute position of the reconnaissance target using GPS (Global Positioning System) and how much range in which direction from that position A second process of synthesizing the virtual image and the real image generated in the first process by obtaining information about the material that is photographing the image; It consists of a third process.

1 is a block diagram of an unmanned reconnaissance system using augmented reality technology according to the present invention,

2 is a flowchart illustrating a method of operating an unmanned reconnaissance system according to the present invention;

3 is a conceptual diagram of a wireless communication exchange according to the present invention;

4 is a conceptual diagram of a photographing information receiving apparatus according to the present invention;

5 is a conceptual diagram of a camera (image transmission) according to the present invention;

6 is a conceptual diagram of a camera position detection (GPS) configuration according to the present invention;

7 is a conceptual diagram of a gyro (camera direction detection) according to the present invention;

8 is a process flow diagram for the real-world and virtual image synthesis according to the present invention,

9 is a process flow diagram for the actual image and virtual image synchronization according to the present invention,

10 is an overall block diagram of an unmanned reconnaissance system according to the present invention;

11 is a process diagram for the main operation of the unmanned reconnaissance system of the present invention,

12 is a process diagram for the operation of the image compositing module according to the present invention;

13 is a process diagram for the operation of the virtual image processing module according to the present invention;

14 is a process diagram for the operation of the photorealistic image processing module according to the present invention;

15 is a process diagram for the operation of the virtual image generation module according to the present invention;

16 is a process diagram for the operation of the map DB search module according to the present invention;

〈Description of the symbols for the main parts of the drawings〉

100: GPS satellite 200: base station

201: personal computer (PC) 202: test map D / B

203: TNC 204: first receiver (RX)

205: second transceiver (TX, RX) 300: helicopter

301: helicopter mounted camera 302: data transmission device (TX)

303: Gyro System 304: GPS Receiver

305: antenna 306: gyro engine

307: gyro test box

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an unmanned reconnaissance system using augmented reality technology according to the present invention, Figure 2 is a process flow diagram for the operation method of the unmanned reconnaissance system according to the present invention, Figure 3 is a wireless communication exchange conceptual diagram according to the present invention 4 is a conceptual diagram of a photographing information receiving apparatus according to the present invention, FIG. 5 is a conceptual diagram of a camera (image transmission) according to the present invention, FIG. 6 is a conceptual diagram of a camera position detection (GPS) according to the present invention, and FIG. 8 is a conceptual diagram of a gyro (camera direction detection) configuration according to the present invention, FIG. 8 is a flowchart illustrating a process for synthesizing real images and virtual images according to the present invention, and FIG. 10 is an overall block diagram of an unmanned reconnaissance system according to the present invention, FIG. 11 is a processing diagram for the main operation of the unmanned reconnaissance system of the present invention, and FIG. FIG. 13 is a processing diagram of the operation of the virtual image processing module according to the present invention, FIG. 14 is a processing diagram of the operation of the real image processing module according to the present invention. 15 is a flowchart illustrating an operation of a virtual image generation module according to the present invention, and FIG. 16 is a flowchart illustrating an operation of a map DB search module according to the present invention.

Referring to FIG. 1, in an unmanned reconnaissance system using augmented reality technology, a helicopter 300 equipped with a CCD camera, a data transmission device (TX, RX), a gyro system, and a camera position detection receiver (GPS) detects 24 camera positions. (GPS) configured to communicate with satellite 100 and base station 200. In this case, the base station 200 is a personal computer 201, a test map database (DB) 202, a terminal node controller (TNC) 203 and a transceiver 204. , 205 is mounted.

The unmanned reconnaissance system of the present invention having the above configuration transmits the live image captured by the helicopter 300 from the helicopter 300 and the location information provided by the GPS receiver through the transceiver TX and RX, and transmits the information to the transceiver. The information received at the base station via 204 and simultaneously synchronized with the test map DB 202 information using the personal computer (PC) 201 is displayed on the screen. Then, the user of the personal computer (PC) 201 performs the reconnaissance process with the information displayed on the screen.

Referring to FIG. 2, in the method of operating the unmanned reconnaissance system according to the present invention, first, to obtain a real image of a reconnaissance target, the reconnaissance target is photographed to collect current information on the reconnaissance target, and the place where data is already recorded on the shooting location. In order to create a virtual image represented by a picture or a letter of information such as the name of the place and the layout of a building or a road, visualize necessary information using computer graphics technology and visualization method (S100), and use a GPS to After obtaining information on the absolute position and in what direction in which direction it is taken, the virtual image generated in the visualization process (S100) is synthesized with the real image (S200), and then the unattended through the user interface. The actual image and the virtual image that change according to the movement of the reconnaissance system are synchronized (S300).

In this case, the weight of the system is 20% in the process of photographing and visualizing (S100), GPS operation and the process of synthesizing the real image and the virtual image (S200), and the real-time and virtual image synchronization process (S300) through the user interface, respectively. 40%.

Referring to FIG. 3, wireless communication in accordance with the present invention is accomplished by a personal computer (PC) 201, a terminal node controller (TNC) 203, and a transceiver 203 '. At this time, the terminal node controller (TNC) 203 outputs an AFSK (******* Full Name *******) output (OUT) and a PPT (******* Full Name *). Transmits through the ******) port, receives through the Squelch serial port and the AUDIO IN port, and the transceiver 203 'connected to the antenna is MIC (******* Full Name *). ******) and PPT ANT Rig (******* Full Name *******) ports to receive Squelch and AF OUT (SP) (******* Full Name *******) communicates through the terminal node controller (TNC) 203 and the transceiver 203 '.

At this time, the personal computer (201) and the terminal node controller (TNC) 203 communicate via RS-232C.

Referring to FIG. 4, photographing information according to the present invention is received through a helicopter-attached camera 301. The received photographing information is attached to a base station 200 through a data transmission device 302 attached to a helicopter. 1 is transmitted to the receiver (GS-550GTH) 204, and via the terminal node controller (TNC) 203 to the personal computer 201 incorporating a printer and a disk via RS-232C.

Referring to FIG. 5, in order to transmit an image according to the present invention, the first receiver (GS-550GTH) in the base station 200 may transmit information captured by the camera with a helicopter 301 through the data transmission device (GS-120R) 302. The first receiver 204 transmits the received information to the personal computer (PC) 201 that calculates the image (TV) and data.

Referring to FIG. 6, in the process of GPS detection of the camera according to the present invention, information transmitted to the GPS receiver 304 in the helicopter 300 is transmitted back to the transceiver through an antenna 305. And transmitted to the terminal node controller (TNC) 203 in the base station 200 through wireless packet communication, and the information received by the terminal node controller (TNC) 203 is transmitted to the personal computer (PC) (RS-232C). 201).

Referring to FIG. 7, a camera direction detection (gyro) according to the present invention first receives direction information from a gyro engine 306 and transmits it to a gyro test box 307. The gyro test box 307 examines the gyro information and transmits the gyro information to the terminal node controller (TNC) 203 in the base station 200 through the transceiver in the helicopter, and transmits the transmitted information to the personal computer through the RS-232C. (PC) 201 is transmitted. Through this process, the direction detection for the camera 301 is performed.

Referring to FIG. 8, in the process for real-time and virtual image synthesis according to the present invention, first, camera information and a live image are acquired (S201), and position information is acquired from camera position detection (GPS) (S202). After analyzing the obtained location information and the photo-realistic information such as the visible range (S203), the virtual information is visualized using the analyzed information (S204), and then the actual image and the virtual image are synthesized from the analyzed information ( S205).

Referring to FIG. 9, in the real time and virtual image synchronization processing according to the present invention, first, camera information and live image information are acquired (S301), position information from a camera position detection (GPS) is obtained (S302), and a change is performed. Information about the factor is obtained (S303). After analyzing the actual information about the position and the visible range of the acquired information (S304), the correction factor according to the change factor of the analyzed actual information is analyzed (S305), and the virtual information using the analyzed information. Visualize (S306). Finally, the real image and the virtual image are synthesized (S307) to process synchronization of information.

Referring to FIG. 10, the main program of the unmanned reconnaissance system according to the present invention is an image synthesizing module 10, and the image synthesizing module 10 includes a virtual image processing module 20 and a live image processing module 30. Then, the virtual image and the real image transmitted from each module are synthesized.

In addition, the virtual image processing module 20 is composed of a virtual image generation module 40 and the map DB search module 50 to process the virtual image generated by these components (40, 50), the live-action The image processing module 30 is composed of a CCD camera control module 70 and a GPS and gyro control module 80 to process the real image obtained by these components 70 and 80.

At this time, the virtual image processing module 20 and the live-action image processing module 30 share a wireless communication control module 60 for controlling communication between each other.

Referring to FIG. 11, the processing for the main operation of the unmanned reconnaissance system of the present invention first sets a window environment (1), sets initial values necessary for the image synthesizing module or less, and then sets an event loop (2). (3) When wireless information such as live image, location, direction, etc. generates an input event, the sub module (image composition module) is called and processed.

Referring to FIG. 12, the processing for the operation of the image synthesizing module according to the present invention obtains initial information 11 of each accessory device, such as a CCD camera, a GPS receiver, a gyro, a wireless transceiver, and manages the system. After initializing the system with the values set in the module (main program) (12), the sub-modules (virtual image processing module and live image processing module) are called (13) to overlay the real and virtual images and combine them (14). Let's do it. When an error event occurs, a correction value for the error value is generated and corrected (15).

Referring to Figure 13, the processing for the operation of the virtual image processing module according to the present invention interprets the input signal received from the wireless receiver (RX) of the base station 21 by the CCD camera direction and field of view (field of view) Find out the location of helicopter, helicopter, map DB, etc., and provide it to the sub-modules (virtual image generation module and map DB search module) (22). After passing to the (virtual image processing module) (23), the information received from the wireless communication control module is obtained (24) to repeat the above.

Referring to Figure 14, the processing of the operation of the real-time image processing module according to the present invention acquires the state of each accessory device, such as CCD camera, GPS receiver, gyro, wireless transceiver, 31 by the information and the CCD camera The acquired due diligence image is notified to the upper module (image composition module) through the wireless communication module (32), and the transmission of such due diligence image is notified to the upper module (33).

Referring to FIG. 15, the processing of the operation of the virtual image generating module according to the present invention may set (41) observer information such as a position, a viewing direction, a field of view, and the like. After changing the location of the information by the observer's eye coordinate to handle the perspective (42), removing the information of the invisible area (invisible building and its place name) (43) and then only the information in the visible area. The virtual image generated by rendering 44 in perspective is provided to the upper module (image composition module).

Referring to FIG. 16, the processing of the operation of the map DB search module according to the present invention analyzes the observer information as shown in FIG. 13 to find map information in the visible region (51), and invisible region information. After removing 52, only the map information in the visible region is provided to the upper module 53 (compositing image).

Table 1 shows the overall arrangement of the implementation of the virtual world representation technology, the implementation of the combined technology of the real world and the virtual world, and the synchronization technology of the real world and the virtual world.

Main Category Category Cattle classification Implement virtual world representation technology ● Virtual object creation ● Development of visualization representation method of real world information -3D modeling technology-virtual reality technology-information visualization expression technology Synchronization between Real and Virtual Worlds ● Synthesis of Real World Image and Virtual World Image -Image processing technology-Position recognition technology-Image overlay expression technology-Display scope expression technology

Table

Realization of Synchronization Technology between Real and Virtual Worlds Synthesis of Real World Coordinates and Virtual World Images Synchronization of Real and Virtual Objects Synchronization of Real and Virtual World Information -Position and orientation recognition and processing technology-Image overlay processing technology-Environment change correction technology-Scalable font processing technology-Display Scope processing technology-Synchronization technology

Referring to Table 1, in order to implement the virtual world representation technology of the present invention, a virtual object was created and a visual representation method of real world information was developed. For this, 3D modeling technology, virtual reality technology, information visualization representation technology, and 3D font design were developed. The technology was used, and in order to realize the combined technology of the real world and the virtual world, we synthesized the real world image and the virtual world image and developed a combination method of the virtual world image according to the change of visible environment. , Image overlay expression technology, font processing technology, and Display Scope expression technology were used. In addition, in order to realize the synchronization technology between the real world and the virtual world, the real world coordinate and the virtual world coordinate synchronization, the real world and the virtual world object synchronization, and the real world and the virtual world information synchronization have been implemented. Image overlay processing technology, environment change correction technology, scalable font processing technology, display scope processing technology, and synchronization technology.

As described above, the method of operating the unmanned aerial vehicle system using the augmented reality of the present invention combines the advantages of human empirical judgment, fast computation of a computer, and spatial and physical non-constraints. The use of information can be increased by facilitating the operation.

Claims (3)

In the unmanned reconnaissance system using Augmented Reality, In order to obtain a realistic image of the reconnaissance target, the reconnaissance target is photographed to collect information on the reconnaissance target, and the information such as the name of the place and the layout of the building or road, which is already data-formed for the shooting location, is used to express pictures or letters. A first step of visualizing the necessary information such as the main building name, the height of the building, the cross-sectional area, and the road name of the place being viewed using computer graphics technology and a visualization method using pictures and texts to generate a virtual image; The virtual image and the actual image generated in the first step are obtained by obtaining information about the absolute position of the object to be scouted and the field of view in the viewing direction from the position using the GPS. The second process of synthesizing the coordinate system (virtual image coordinate system and photorealistic image coordinate system) with the same image overlay technology; As the unmanned reconnaissance system moves through a user interface, the image of the position, the viewing direction, the field of view, etc. is automatically changed and the virtual image is considered in consideration of the changed position, the direction, and the visible area. And a third process of overlaying two images to achieve image synchronization. The method of claim 1, wherein the second process A first step of obtaining location information of a reconnaissance target from CCD camera information, photorealistic images, and GPS information; A second step of analyzing the acquired position information and the live-action information of the visible range; Visualizing virtual information using the interpreted information; And a fourth step of synthesizing the actual image and the virtual image from the interpreted information. The method of claim 1, wherein the third process is A first step of obtaining location information and change information from the camera information, the actual image, the GPS; A second step of parsing due diligence information on the position and the visible range of the obtained information; A third step of analyzing a correction factor according to the change factor of the analyzed due diligence information; A fourth step of visualizing the virtual information using the interpreted information; And a fifth step of synthesizing the real image and the virtual image after visualizing the analyzed information.