KR102161917B1 - Information Processing System and method for rescue in mountain area using UAS - Google Patents

Abstract

The present invention relates to an information processing system using an unmanned aerial vehicle for rescue in a mountain area and a method thereof. The information processing system comprises: an unmanned aerial vehicle that analyzes image information obtained from at least one camera while flying over an arbitrary area to determine whether the arbitrary area is a disaster area or whether there is a rescuee, and detects the position of the rescuee when it is determined that there is the rescuee; a rescuer terminal that transmits real-time position information of the rescuer; and a control server that receives image information and position information of the rescuee and the real-time position information of the rescuer, sets a rescue route of the rescuer and an evacuation route of the rescuee according to a presorted rescue scenario so that the rescuer and the rescuee can meet each other, provides the rescue route to the rescuer terminal, and provides the evacuation route to the unmanned aerial vehicle, wherein the control server updates the position information of the rescuee and the position information of the rescuer, correct the rescue route and the evacuation route based on the updated information, and the unmanned aerial vehicle guide flies along the evacuation route.

Description

Information processing system and method for rescue in mountain area using UAS}

The present invention relates to an information processing system and method for the rescue of a mountainous area using an unmanned aerial vehicle, and more particularly, in the event of a disaster situation, a rescuer can be quickly rescued by using an unmanned aerial vehicle that unmannedly flies over a preset area. It relates to an information processing system and a method for the rescue of a mountainous area using an unmanned aerial vehicle.

Most of Korea's land is mountains, and initial response is important in the event of a disaster caused by forest fires, landslides, and heavy rain.However, real-time disaster information due to inadequate systematic system construction due to limitations in operating power supply, etc. As it is difficult to obtain, it is difficult to quickly judge the situation and respond.

In addition, it is difficult to provide disaster broadcasting and evacuation information to citizens isolated in mountainous areas or located in the shadow areas of disaster broadcasting, making it difficult to secure safety and rescue for citizens.

On the other hand, drone is a word that has a dictionary meaning of bee buzzing or low buzzing, and it is also expressed as an unmanned aerial vehicle (UAV) in that it is remotely controlled from the ground without a person riding on the aircraft. do. In other words, a drone is an unmanned aerial vehicle that can fly automatically or remotely without a pilot on board the aircraft, and can be used once or retrieved.

Initially, drones were mostly used for military purposes, but with continuous technological development, they are currently being used in private sectors such as weather observation, environmental and forest fire monitoring, border/coast/road monitoring, disaster support communication relay, and remote sensing.

The present invention was conceived to solve the above-described problem, and when using an unmanned aerial vehicle to rescue a rescuer located in an area where access is difficult in the event of a disaster in a mountainous area, the information transmission period is adaptively controlled and It provides a system and method for efficiently using a battery.

In addition, it is intended to easily obtain information on the situation of rescuers in the event of a disaster in a mountainous area.

In order to achieve the above object, an information processing system for the structure of a mountainous area using an unmanned aerial vehicle according to an embodiment presented in the present specification analyzes image information acquired from one or more cameras while flying in an arbitrary area. Thus, the unmanned aerial vehicle which determines whether the flight area is a disaster area and whether a rescuer exists, and detects the location of the rescuer when it is determined that the flight area is a disaster area and a rescuer exists; A rescuer terminal for transmitting its own real-time location information; And receiving image information and location information of the rescuer and real-time location information of the rescuer terminal, setting a rescue route and an evacuation route of the rescuer, providing a rescue route to the rescuer terminal, and an evacuation route to the unmanned aerial vehicle It includes a control server that provides a.

The control server updates the rescuer's location information and the rescuer's location information to correct the rescue route and the evacuation route based on the updated information, and the unmanned aerial vehicle guides the flight along the evacuation route.

The control server calculates the distance between the rescuer and the rescuer based on the rescuer location information and the rescuer location information, sets a transmission period having a positive correlation with the calculated distance, and sets the set transmission period to the unattended It transmits to the vehicle, and the unmanned aerial vehicle transmits image information and location information to the control server according to the transmission period.

The control server transmits, to the unmanned aerial vehicle, disaster evacuation information including one or more of a disaster occurrence warning and evacuation instructions based on the location and image information of the rescuer.

The unmanned aerial vehicle transmits a message including disaster evacuation information received from the control server through a speaker.

The unmanned aerial vehicle includes a microphone and a speaker, and determines whether it is necessary to transmit an alarm broadcast based on the sound information and the image information acquired from the microphone, and transmits the disaster alarm broadcast through a speaker accordingly.

The method of the information processing system for the structure of a mountainous area using an unmanned aerial vehicle according to an embodiment presented in the present specification is, while the unmanned aerial vehicle flies in an arbitrary mountainous area, acquires image information from one or more cameras, and Analyzing the image information to determine whether a disaster has occurred in the flight area and whether a rescuer exists; If the unmanned aerial vehicle determines that a rescuer exists, detecting the location of the rescuer, and transmitting the detected location information and image information of the detected location to one or more of a pre-registered rescuer terminal and the control server; Receiving, by a control server, image information and location information of the rescuer and real-time location information of the rescuer terminal, and setting a rescue route and an evacuation route of the rescuer according to a previously stored rescue scenario; The control server providing a rescue route to the rescuer terminal and providing an evacuation route to the unmanned aerial vehicle; And guided flight by the unmanned aerial vehicle according to the evacuation route.

The control server updates the location information of the rescuer and the location information of the rescuer, and corrects the rescue route and the evacuation route based on the updated information.

In the method of the information processing system for the rescue of a mountainous area using an unmanned aerial vehicle, a control server calculates a distance between the rescuer and the rescuer based on the rescuer location information and the rescuer location information, and the calculated distance and amount Setting a transmission period having a correlation and transmitting the set transmission period to the unmanned aerial vehicle; And transmitting, by the unmanned aerial vehicle, image information and location information to the control server according to the transmission period.

The control server transmits, to the unmanned aerial vehicle, disaster evacuation information including one or more of a disaster occurrence warning and evacuation instructions based on the location and image information of the rescuer.

The unmanned aerial vehicle transmits a message including disaster evacuation information received from the control server through a speaker.

A method of an information processing system for the rescue of a mountainous area using an unmanned aerial vehicle, comprising: the control server partitioning the mountainous area into a plurality of areas and setting a disaster urgency level for each of the divided areas; And providing, by the control server, different disaster warning information according to the urgency level to an alarm device of a matching location.

The rescue system in a mountainous area using the unmanned aerial vehicle of the present invention as described above facilitates relief of a rescuer located in a difficult-to-access area.

In addition, it is possible to easily obtain information on the situation of a rescuer in the event of a disaster situation in a mountainous area.

1 is a diagram schematically illustrating an information processing system for a structure of a mountainous area using an unmanned aerial vehicle according to an embodiment of the present invention.
2 is a view schematically illustrating the configuration of an unmanned aerial vehicle according to an embodiment of the present invention.
3 is a diagram schematically illustrating the configuration of a control unit of an unmanned aerial vehicle according to an embodiment of the present invention.
4 is a diagram schematically illustrating a configuration of a control server according to an embodiment of the present invention.
5 is a diagram schematically illustrating a configuration of a rescuer terminal according to an embodiment of the present invention.
6 is a flowchart illustrating an integrated rescue method for a mountainous area using an unmanned aerial vehicle according to an embodiment of the present invention.
7 is a flowchart illustrating a method of checking whether a disaster occurs in an unmanned aerial vehicle according to an embodiment of the present invention.

It should be noted that technical terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present specification should be interpreted as generally understood by those of ordinary skill in the technical field to which the present invention belongs, unless otherwise defined in the present specification, and excessively comprehensive It should not be construed as a human meaning or an excessively reduced meaning. In addition, when a technical term used in the present specification is an incorrect technical term that does not accurately express the spirit of the present invention, it will be replaced with a technical term that can be correctly understood by those skilled in the art to be understood. In addition, general terms used in the present invention should be interpreted as defined in the dictionary or according to the context before and after, and should not be interpreted as an excessively reduced meaning.

In addition, the singular expression used in the present specification includes a plurality of expressions unless the context clearly indicates otherwise. In the present application, terms such as "consist of" or "include" should not be construed as necessarily including all of the various elements or various steps described in the specification, and some of the elements or some steps It may not be included, or it should be interpreted that it may further include additional elements or steps.

In addition, the suffixes "module" and "unit" for the constituent elements used in the present specification are given or used interchangeably in consideration of only the ease of writing the specification, and do not themselves have a distinct meaning or role from each other.

In addition, terms including ordinal numbers such as first and second used in the present specification may be used to describe various elements, but the elements should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted.

In addition, in describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are intended to make the spirit of the present invention easier to understand, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

1 is a diagram schematically illustrating an information processing system for a structure of a mountainous area using an unmanned aerial vehicle according to an embodiment of the present invention.

As shown in FIG. 1, the alarm broadcasting system in the shaded area using an unmanned aerial vehicle according to an embodiment of the present invention includes an unmanned aerial vehicle 100, a remote control device 200, a control server 300, and an alarm device 400. And it may include a terminal 500.

The unmanned aerial vehicle 100 is formed to be maneuverable by remote operation without a person's boarding. The unmanned aerial vehicle 100 is connected to the remote control device 200 through a communication link for control. The unmanned aerial vehicle 100 has its own address, for example a MAC address, used for communication with the remote control device 200.

The remote control device 200 receives the MAC address of the unmanned aerial vehicle 100 to identify an arbitrary unmanned aerial vehicle, and provides a moving path for autonomous driving of the unmanned aerial vehicle 100 through a communication link or unmanned aerial vehicle 100 ) Directly control the driving.

The control server 300 determines whether or not a disaster has occurred in a mountain area based on the disaster-related information received from an external device, and transmits a disaster alert broadcast according to the disaster-related information through the VHF communication network.

The control server 300 transmits and receives information through the unmanned aerial vehicle 100 and a communication network. For example, the control server 300 receives real-time flight information such as location information, attitude information, and altitude information of the unmanned aerial vehicle 100, and, if necessary, transmits operation commands related to the flight control of the unmanned aerial vehicle 100 to a wireless communication network. To be transmitted to the unmanned aerial vehicle 100 through. The operation command related to flight control may include any one or more of setting a broadcast point and changing a flight path.

That is, the control server 300 identifies the unmanned aerial vehicle 100 in flight, and checks the current flight position of the identified unmanned aerial vehicle 100. In addition, the control server 300 detects the movement path of the unmanned aerial vehicle, analyzes the image received from the unmanned aerial vehicle 100, and transmits an operation command related to flight control.

The control server 300 processes image data, audio data, and sensing data, which are collected information received from the unmanned aerial vehicle 100. In addition, the control server 300 may directly transmit real-time alarm broadcasting information to the unmanned aerial vehicle 100. The real-time alarm broadcast information may be transmitted through a mobile communication network such as LTE, but is not limited thereto.

The control server 300 receives disaster information from a central control center that manages disaster information, transmits it to the unmanned aerial vehicle 100, or transmits disaster broadcast information input in real time through a microphone provided in the control server 300. Can be sent to (100). The control server 300 may determine a range of a disaster occurrence site based on a geographic information system (GIS) of a disaster site, and integrate and manage one or more alarm devices 400 distributed and installed in the disaster site range. To this end, the control server 300 pre-stores location information in which the alarm device 400 is installed.

The control server 300 is interlocked with a weather forecasting and warning system, a flood monitoring system, a dam water level monitoring system and an earthquake disaster prevention management system, and collects external environment information including meteorological information on an hourly basis.

The control server 300 may perform complex disaster information analysis through data mining on various data, images, videos, etc. including collected sensing information and image information. In addition, ontology-based context awareness is performed to closely analyze the disaster situation and regional characteristics of each location to determine the scope of the disaster occurrence site. In addition, by analyzing the image information, the area of the disaster occurrence, the rate of spread, and the direction of the spread are predicted, and the degree of the urgency of the spread of damage by disaster occurrence area, the moving path of the rapid response organization, and the evacuation path are analyzed. For example, it is possible to calculate the route of movement of the response organization and the route of evacuation of rescuers by avoiding the place with severe flames, the place where the collapse occurred, and the place with severe flooding.

The control server 300 may also broadcast a quick evacuation method and response plan including the calculated guidance route through the unmanned aerial vehicle 100 and the alarm device 400.

The control server 300 calculates an evacuation route and a rescue route according to a pre-stored rescue scenario based on the location of the rescue terminal 500 and the location information of the rescuer obtained from the rescue terminal 500 to be described later, and updates the location information. Correct the evacuation route and rescue route accordingly. Rescuers and rescuers can meet along the rescue route and the evacuation route.

The control server 300 changes the location information update period and/or the evacuation route and the correction period of the rescue route based on the location of the rescue team terminal 500 and the location information of the rescuer obtained from the rescue team terminal 500. For example, the control server 300 may shorten a location information update period and/or a correction period for an evacuation route and a rescue route as the rescue team terminal 500 and the location of a rescuer get closer.

The control server 300 may shorten the update period of location information and/or the correction period of the evacuation route and the rescue route in an area in which the number of cases of the rescuer's movement route is greater than otherwise. For example, when a rescuer is located on multiple roads, the location information update period (and/or a correction period of the route) may be shorter than when the rescuer is on a dead end.

The alarm device 400 is provided in one or more places in a mountainous area, receives a disaster alert broadcast from the control server 300 through a VHF communication network, and transmits the disaster alert broadcast to the outside through a broadcast facility. The broadcast facility may be a plurality of alarm siren devices.

The alarm device 400 may be configured as, for example, a box-shaped enclosure, and includes an internal memory and an external memory, and includes a UART (Serial) connection terminal, a USB connection terminal, and a digital I/O. On the front of the box-shaped enclosure, LEDs that display equipment status, transmission path status, operation status, and amplifier output status are provided, operation switches such as microphone transfer switches are provided, and A/S RS-232C port, USB port, A microphone connection terminal, an external memory connection terminal, a microphone volume control terminal, a monitor speaker volume control terminal, and a speaker are provided.

On the back of the box-shaped enclosure, AC power input terminal, On/Off switch, speaker output terminal, remote (REMOTE) VHF antenna, local (LOCAL) VHF antenna, Digital I/O terminal, audio LINE OUT terminal, LAN cable terminal, DC An output terminal or the like is provided.

In addition, the alarm device 400 further includes an amplifier to amplify the audio output of the disaster alert broadcast received from the control server 300 to transmit audio to a predetermined transmission range. The amplifier may be a digital amplifier using a digital chip without an analog circuit. Digital amplifiers are smaller than analog amplifiers and generate less heat.

The rescuer terminal 500 is a terminal capable of receiving and outputting information from the control server 300 and the unmanned aerial vehicle 100 through a communication network, and receives rescue request information from the control server 300 and the unmanned aerial vehicle 100 , It can be provided to disaster relief personnel such as firefighters, police officers, and paramedics performing rescue, or can be provided in the situation room of each related organization such as fire station and police station.

These terminals have high strength, are strong against impact, prevent the penetration of fire heat, and are preferably flameproofed. Rescuers may use the rescuer terminal 500 to transmit their own situation information, information on a rescue site, and information on a person to be rescued. Here, the situation information includes any one or more of location information, body change situation, and environment information of the rescuer, and may include any one or more of age, sex, and body change situation of the rescue target.

The control server 300 and the unmanned aerial vehicle 100 previously store contact information such as the IP address or mobile communication number of the terminal 500 in order to transmit information to the terminal 500.

The rescuer terminal 500 may be, for example, a mobile phone, a smart phone, personal digital assistants (PDA), a personal computer (PC), a tablet personal computer (PC), and a notebook of various types, such as a notebook.

The communication network is TCP/IP, LAN (Local Area Network), WIFI, LTE (Long Term Evolution), WCDMA (Wideband Code Division Multiple Access), other known or future wired communication, wireless communication methods, and other communication methods. It can be implemented using at least some. Although a lot of communication is performed through a network, in the description to be described later, a reference to the network is omitted for brevity.

FIG. 2 is a diagram schematically illustrating a configuration of an unmanned aerial vehicle according to an embodiment of the present invention, and FIG. 3 is a view schematically illustrating a configuration of a controller of an unmanned aerial vehicle according to an embodiment of the present invention.

The unmanned aerial vehicle 100 includes a wireless transmission/reception unit 110 for transmitting and receiving a wireless signal, a control unit 120 for performing various controls, a position receiving unit 130 for receiving a position signal, an image capturing unit 140 for photographing an image, The message generator 150 may include an alarm output unit 160 and a data storage unit 190 for outputting an alarm for a disaster situation, and a power supply unit 180 for supplying power to each unit.

In addition, the unmanned aerial vehicle 100 may further include a grab module unit 170 equipped with a disaster relief device and a fall prevention unit 175 for preventing a fall to the ground.

The wireless transmission/reception unit 110 transmits and receives signals to and from the remote control device 200 or the control server 300.

The wireless transmission/reception unit 110 transmits the captured still image or video data, audio data, and sensing data to the control server 300, and receives an operation command related to the flight control of the unmanned aerial vehicle 100 from the control server 300 Receive. The wireless transmission/reception unit 110 transmits the received information to the control unit 120.

In the unmanned aerial vehicle 100, a transmission/reception module used for communication with the remote control device 200 and a transmission/reception module used for communication with the control server 300 may be different from each other. For example, communication with the remote control device 200 may use a 2.1 GHz frequency, and communication with the control server 300 may use a 108 GHz frequency.

The controller 120 is a module that controls a series of processes related to the transmission/reception of operation commands for receiving control from the control server 100 and processing according to the control when the unmanned aerial vehicle 100 is in flight.

In addition, the controller 120 detects flight postures such as take-off and landing and navigation of the unmanned aerial vehicle 100 and controls the flight information to be transmitted to the control server 300. The controller 120 may be configured to include an acceleration sensor, a gyro sensor, an altitude sensor, a geomagnetic sensor, and the like for flight control. In this way, the controller 120 is in charge of controlling the execution process when performing a series of functions such as providing altitude, posture, and location information during flight of the unmanned aerial vehicle 100.

The location receiving unit 130 includes a Global Position System (GPS) module and acquires location information of the unmanned aerial vehicle 100. The GPS module calculates information about a distance from three or more satellites and a time at which the distance information is measured, and then applies a trigonometry to the calculated distance information. It is possible to calculate three-dimensional location information according to latitude, longitude, and altitude for a point (object). Further, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is also used.

The location receiving unit 130 transmits the acquired location information to the control unit 120. The control unit 120 may match the image acquired by the image capturing unit 140 with the location information and store it in the data storage unit 190.

The image capture unit 140 includes a camera for capturing an image, and includes a visible ray camera, a thermal imaging camera, a hyperspectral sensor, a light detection and ranging (LIDAR), and a synthetic aperture radar (SAR). Including, and may further include a microphone (microphone) 142.

Visible light cameras are used for detection of flames and smoke that can be detected in the visible light band, detection of submerged areas, and detection of changes in the surface of the earth. Still images and videos are acquired and stored.

Thermal imaging cameras are devices that track, detect, and display heat using infrared wavelengths (0.9-14㎛). Thermal imaging cameras display relative or absolute temperature values in black and white and output them.

The image capturing unit 140 may collect audio data using only a microphone (microphone) separated from image capturing. The image capturing unit 140 transfers the captured image to the control unit 120, and the control unit 120 performs processing such as image correction, and then stores it in the data storage unit 190.

The image captured by the image capturing unit 140 may be provided together with location information and time information acquired by the position receiving unit 130.

The message generator 150 generates a message including the location information of the helper. The generated message is transmitted to the terminal 500 or the control server 300 registered in advance according to the instruction of the control unit 120.

Along with the message, image information of a rescuer may be transmitted together.

The alarm output unit 160 may output a pre-stored alarm through a siren or a warning lamp, and may receive disaster broadcasting and disaster situation information in real time from a control server and output through a speaker.

The power supply unit 180 supplies power to each component. The power supply unit 180 may be a hybrid type including a DC power source 182 and a solar module 184 that generates power using sunlight.

The controller 120 monitors the remaining amount of the battery of the power supply unit 180.

The grab module unit 170 is equipped with equipment for life-saving, such as first aid supplies, gas masks, life-saving tubes, and simple fire extinguishers.

The control unit 120 controls the grab module unit 170 to provide mounted rescue equipment to the rescuer.

The fall prevention unit 175 is provided to unfold a balloon such as helium gas, and is implemented to operate when a driving signal is output from the control unit 120.

In addition, the fall prevention unit 175 is opened in the form of a parachute when a fall situation of the unmanned aerial vehicle 100 occurs, so that the impact is minimized when the unmanned aerial vehicle 100 falls and touches the ground.

The power supply unit 180 may be a hybrid type including a DC power source 182 and a solar module 184 that generates power using sunlight.

The data storage unit 190 stores programs and data necessary for an operation of analyzing image information and transmitting a message including location information and/or image information, and may be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the automatic flight, a program for checking the remaining battery level, data necessary for generating and transmitting a message (eg, a contact information of an information receiving terminal) and a program. The data area is an area in which information generated by an operation is stored, and may store information generated by an operation, such as image information generated by photographing and location acquisition information, and the like.

Referring to FIG. 3, the controller 120 can be divided into a flight control module 122, an image processing module 124, a message generation module 126, and an alarm broadcasting module 128. Each module is functionally classified for convenience of explanation, but may be performed as a single module or may be integrated in different ways.

The flight control module 122 detects flight attitudes such as take-off and landing and navigation of the unmanned aerial vehicle 100 and controls the flight information to be transmitted to the control server 300, and for flight control, an acceleration sensor, a gyro sensor, an altitude sensor, It may be configured to include a geomagnetic sensor or the like.

The flight control module 122 may automatically fly avoiding the obstacle by changing an altitude, for example, when an obstacle is detected through a distance sensor based on the signal from the sensor. In another modified example, it is of course possible to perform obstacle avoidance control through the remote control device 200 or the control server 300.

The flight control module 122 may control to reduce the speed or lower the altitude at the rescuer discovery position when a rescuer is found according to the image processing module 124 to be described later.

In addition, the flight control module 122 calculates the distance to the preset return place, calculates the amount of power required to return to the return place, and a value excluding the amount of power required to return from the remaining battery power is a preset first threshold value. If it is below, it switches from normal mode to battery saving mode.

The battery saving mode may include any one or more of stopping camera video recording, stopping video information transmission, and stopping alarm broadcasting. Alternatively, depending on the remaining battery capacity, camera video recording may be stopped, image information transmission may be stopped, and alarm broadcast may be stopped in sequence.

In addition, the flight control module 122 drives the fall prevention unit 180 when the value excluding the amount of energy required for return from the remaining battery capacity is less than or equal to a preset second threshold value, so that the operation of the propeller of the wireless vehicle 100 is Even if stopped, it prevents the unmanned aerial vehicle 100 from falling to the ground due to the buoyancy of the balloon filled with helium gas.

In addition, the flight control module 122 controls the grab module unit 170 to fly to a corresponding position of a victim isolated due to a forest fire and provide first aid supplies, gas masks, and a simple fire extinguisher. You can fly to that location and provide life-saving tubes.

The image processing module 124 includes a camera and a codec, and may obtain an encoded image by encoding a raw image captured from the camera with a codec such as MJPEC, MPEG, and HEVC, removing noise from the obtained encoded image, and adjusting the scale. A pre-processing process such as performing a correction for distortion and the like is performed.

For example, when the lens of the camera is a wide-angle lens, radiation distortion inevitably caused by the wide-angle is corrected. Temperature correction can be easily performed by multiplying the temperature value of the coordinates by the area ratio of the coordinates based on the area ratio for each coordinate system previously stored in the original image of the thermal image where distortion occurs. Compensating the temperature by applying the area ratio for each coordinate stored in advance in this way is not large, so it can be performed within the drone itself. Image correction can be similarly performed.

The image processing module 124 may detect an image feature point and a region of interest (ROI) from image information on which a pre-processing process has been performed, and then perform analysis such as detection of a person and occurrence of a disaster. Person detection, the image processing module 124 is performed by a method of identifying a person and a fire by comparing the temperature analysis result through the image and image that includes an infrared ray in the captured image with pre-stored reference data. I can.

The determination as to correspond to a disaster situation may analyze the image data by at least one analysis method among machine learning-based analysis, a combination analysis of a plurality of image data, and a linkage analysis with sensing data.

In addition, image data may be analyzed using image data at a location corresponding to the audio data, and whether a disaster has occurred may be determined using the analysis result of the image data.

For example, in image information, a foreground image including an object (eg, flame, smoke, etc.) to be monitored may be separated from a background image excluding the foreground image according to a long-term background modeling method. The object included in the separated foreground image is recognized by referring to previously stored pattern information. When information about a temperature range is set in the pattern information, the temperature of each object is classified based on brightness information according to the temperature included in the thermal image data. Therefore, among the plurality of objects included in the foreground image, it is possible to distinguish an object of smoke from an object of similar image such as cloud or fog. As described above, objects that do not belong to the temperature range included in the pattern information are subjected to background processing, thereby reducing the amount of computation generated by subsequent analysis of continuous thermal image data.

On the other hand, when extracting an object for smoke considering only the temperature range, there may be a case in which the temperature of the cloud or fog rises temporarily or the temperature of the car satisfies the temperature range set in the pattern information. For this purpose, the pattern information may include information on a movement direction or diffusion, which is an inherent property of smoke. That is, smoke generally moves upward or diagonally upwards, and has a characteristic of spreading, so that it can be stored as pattern information to further distinguish it from other objects included in the foreground image. In this case, the movement direction may be determined based on the number of pixels and the movement vector of the pixels constituting the object.

Accordingly, in order to exclude other objects except smoke, an object is tracked based on pattern information, and an object that satisfies a condition set in the pattern information can be detected by checking a change in the number of pixels or a moving direction for each extracted object. Accordingly, it is possible to accurately recognize a desired object from the image information.

The image processing module 124 is configured to transmit, to the control server 100, information acquired when a disaster or a person in distress occurs, such as a photographed image and photographing location information, according to a result of image analysis.

The message generating module 126 checks the point at which the helper is recognized according to the analysis result of the image processing module 124 and obtains the location information of the helper, and stores a message including the obtained location information in advance, and It transmits to at least one of the control server 300.

The alarm broadcasting module 128 outputs a pre-stored alarm broadcasting according to the result of the image processing module 124.

For example, when it is determined that a disaster such as a forest fire has occurred as a result of analysis by the image processing module 124, a pre-stored alarm broadcast matching the forest fire is output through the alarm output unit 160.

In addition, when real-time alarm broadcasting is received from the control server 300, the alarm broadcasting module 128 preferentially outputs the real-time alarm broadcasting through the alarm output unit 160.

In addition, the unmanned aerial vehicle 100 may be equipped with an environmental sensor that collects information on temperature and humidity, wind direction and wind speed, pollution, and the like, and further measure carbon dioxide, carbon monoxide, and the remaining amount of the reservoir by such sensor May be.

The unmanned aerial vehicle 100 may store the sensed information in the data storage unit 190 or transmit it to the control server 300.

4 is a diagram schematically illustrating a configuration of a control server according to an embodiment of the present invention.

The control server 300 may include a communication unit 310, a control unit 330, a storage unit 350, a broadcast transmission unit 370, a display unit 380 and an input unit 390.

The communication unit 310 receives information from an external device (not shown) and the unmanned aerial vehicle 100, and transmits the received information to the control unit 330.

The control unit 330 controls the overall operation of the control server 300 to rescue a rescuer.

The control unit 330 analyzes the disaster situation based on the information received through the communication unit 310 and the previously stored analysis-related database. The controller 330 selects different response messages for the analyzed disaster situation. A plurality of correspondence messages are pre-stored in the storage unit 350, and an arbitrary correspondence message is selected based on the analysis result and location information.

The controller 330 may calculate a guidance route based on pre-stored topographic information of a mountainous area, disaster area information based on image analysis, location information of a rescuer, location information of a rescuer, and a location of a disaster such as a forest fire.

The control unit 330 generates a response message including one or more of an induction route, a disaster warning, and an evacuation method, and transmits the response message to the unmanned aerial vehicle 100 through the communication unit 310.

The storage unit 350 stores data under the control of the controller 330 and transmits the requested data to the controller 330. The storage unit 350 may be implemented as a total of physically separated storage devices. When the storage unit 350 is implemented as a total of several devices that are physically separated, communication between several devices may be required. Here, for the sake of simplicity, description will be made on the assumption that the storage unit 350 is implemented as one object.

The broadcast transmission unit 370 broadcasts a disaster alert broadcast according to whether a disaster has occurred or not through a VHF communication network.

The display unit 380 serves to output arbitrary information according to an instruction of the control unit 330. For example, the display unit 370 may display locations of an unmanned aerial vehicle, a rescuer, and a rescuer according to an instruction of the control unit 330.

The display unit 380 may be a separate display device or may be integrally formed with the control server 300. The display unit 370 may be implemented as, for example, a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a projector, and other display devices that are possible in the present, in the past, or in the future. The display unit 380 may display, for example, an interface page for providing information or an information providing result page. Depending on the embodiment, the display unit 380 may be used instead of the display unit 380, instead of the display unit 380, using a component that uses a voice output, vibration, or other method capable of transmitting information to the user.

The input unit 390 serves to transmit input information to the control unit 330. The input unit 390 may include one or more of a key or a microphone. The input unit 390 may be configured integrally with the control server 300 or may be configured as a separate component. The user can input corresponding information in real time through the input unit 390. For example, the user may check image information of the unmanned aerial vehicle 100 in real time, input corresponding information through a microphone, and transmit the corresponding information to the unmanned aerial vehicle 100.

Also, the display unit 380 and the input unit 390 may be integrally formed such as a touch pad.

5 is a diagram schematically illustrating a configuration of a rescuer terminal according to an embodiment of the present invention.

As can be seen with reference to FIG. 5, the rescuer terminal 500 is a portable communication terminal, and includes a communication unit 510, a control unit 520, an input unit 530, a display unit 540, and a location information generation unit 550. And a storage unit 560.

The communication unit 510 exchanges data with the control server 300 and/or other external devices. The communication unit 510 transmits the data received from the control server 300 to the control unit 520. In addition, the communication unit 510 transmits data to the control server 300 under the control of the control unit 520. The communication technology used by the communication unit 510 may vary depending on the type of communication network or other circumstances.

The controller 520 controls the overall operation of the rescuer terminal 500.

The control unit 520 plays a role of outputting information received from the control server 300 or transmitting information obtained from an input unit or a sensor to the control server 300.

The input unit 530 converts a user's input operation into an input signal and transmits it to the control unit 520. The input unit 530 may be implemented as, for example, a keyboard, a mouse, a touch sensor on a touch screen, a touch pad, a keypad, a voice input, and other input processing devices that are possible in the present, in the past, or in the future. The input unit 530 may receive, for example, a user's information provision request input and transmit it to the control unit 520.

The display unit 540 outputs a screen under the control of the controller 520. The display unit 540 may be implemented as, for example, an LCD (liquid crystal display), an LED (light emitting diode), an OLED (organic light emitting diode), a projector, and other display devices that are possible in the present, in the past, or in the future. The display unit 540 may display, for example, an interface page for providing information or an information providing result page. Depending on the embodiment, the display unit 540 may be used instead of the display unit 540 instead of the display unit 540 using a component that uses a voice output, vibration, or other method capable of transmitting information to the user.

The display unit 540 may display its own location, the location of the unmanned aerial vehicle, and the location of a rescuer on the map information of the mountain area.

The location information generation unit 550 includes a GPS (Global Position System) module, and receives location information from GPS satellites, in a shaded area, receives location information through a gyro sensor and a beacon based on the received GPS location coordinates. can do.

The location information generation unit 550 transmits the acquired location information to the control unit 520. The control unit 520 may transmit the location information of the rescuer terminal 500 to the unmanned aerial vehicle 100 directly or through the control server 300 in real time or periodically. In this case, ID (identification information) and time information uniquely assigned to each terminal can be transmitted together.

The storage unit 560 stores data under the control of the controller 520 and transmits the requested data to the controller 520.

When the terminal 500 transmits and receives data, it may be expressed that the communication unit 510 transmits and receives data according to the control of the controller 520 depending on the viewpoint, or that the controller 520 controls the communication unit 510 to transmit and receive data. You can also express it.

In addition, the terminal 500 may include various sensors. The sensor may include, for example, a biometric sensor, a pollution sensor, a temperature and humidity sensor, an infrared sensor, and the like.

Here, the biometric sensor may be a sensor that measures one or more of a biosignal of a rescuer, that is, pulse, body temperature, electrocardiogram, and blood flow.

Pollution sensor and temperature and humidity sensor measure the air pollution level, temperature and humidity in the vicinity of rescue personnel. It is also possible to measure the amount of carbon dioxide, carbon monoxide, and reservoir remaining as well as the level of air pollution in the surroundings. The infrared sensor can acquire an image through an infrared lens near the rescuer.

An information processing method for the structure of a mountainous area using the unmanned aerial vehicle described above will be described later with reference to FIGS. 6 to 7.

6 is a flowchart illustrating an integrated rescue method for a mountainous area using an unmanned aerial vehicle according to an embodiment of the present invention.

According to another embodiment of the present invention, the unmanned aerial vehicle acquires an image from a camera while flying in a mountainous area (S210). The unmanned aerial vehicle may receive and store a flight path from a control server prior to flight. In addition, the flight path can be changed or modified by the control server or remote control device even during flight.

The unmanned aerial vehicle analyzes the image information to determine whether a disaster has occurred in the flight area and whether a rescuer exists (S220). When a plurality of specific values extracted from the image information are compared with a reference specific value of a person (rescuer) stored in advance, if the two values match, it may be determined as a person.

In step S220, if the unmanned aerial vehicle determines that a rescuer exists, the location of the rescuer is detected (S230). In step S220, if the unmanned aerial vehicle determines that a rescuer does not exist, it returns to step S210.

The detected location information and image information of the detected location are transmitted to one or more of the previously registered terminal and the control server (S240).

The control server sets a rescue route and an evacuation route based on the image information, the location information of the rescuer and the rescuer, and transmits it to the unmanned aerial vehicle 100 (S250).

The control server generates response information in consideration of the type of disaster, the extent of the disaster, the expected range of damage, and the evacuation route based on the video information and the location information of the rescuer.

In another variation, the control server may receive weather information from an external device every predetermined time period. The external device may be a server and a terminal of a related institution such as the Meteorological Administration, the earthquake disaster prevention management office, and the flood monitoring office. The weather information includes one or more of wind direction, wind speed, rainfall, snowfall, and direct snow.

To this end, complex analysis can be performed through data mining on videos and the like. In addition, it may be configured to perform an ontology-based context awareness to closely analyze the disaster situation and regional characteristics of each location.

The control server can predict the direction and extent of the disaster damage based on the collected information. For example, the control server divides the mountain area into a plurality of areas, and sets an urgency level for each area according to the expected result. The urgency class can be divided into a plurality of classes, such as an accident area where a disaster occurs, an emergency evacuation area that requires urgent evacuation, a state area that does not require urgent evacuation but requires attention, and a safety area area.

The control server sets different disaster warning information according to the urgency level and provides it to the alarm device and the unmanned aerial vehicle at the matching location. For example, alarm devices and unmanned aerial vehicles located at different distances from a disaster area broadcast different countermeasures.

In addition, the control server can display the location of the disaster occurrence, the predicted damage range, and the location of a rescuer on the map of the disaster occurrence area and output it to the display unit. For example, a plurality of photos or videos taken while an unmanned aerial vehicle orbiting or moving around a disaster site can be taken at different angles, and at this time, the control server receiving the captured image information will continuously capture the images in a panorama format. It can be synthesized. Through this, the manager of the control server (for example, a disaster expert) can predict the number and distribution of rescuers.

The control server may synthesize a plurality of image information received from the unmanned aerial vehicle in a panorama format, and display the moving position of the rescuer on a map.

Through this, experts of the control server can directly input response information. The information entered by experts of the control server can be provided as sound information through a voice conversion function by a text to speech (TTS) system.

In another example, response information in a voice format input through a microphone by experts may be transmitted to the unmanned aerial vehicle through a real-time mobile communication network or the like. In this case, it may be easier for the helper to respond in real time.

In addition, the control server can also transmit evacuation instructions and countermeasures.

The unmanned aerial vehicle guides flight according to the evacuation route (S260). In addition, the unmanned aerial vehicle 100 outputs the corresponding information received from the control server through the speaker. The unmanned aerial vehicle turns on a beacon during a guided flight of an evacuation route so that a rescuer can easily move along the unmanned aerial vehicle.

The control server updates the location information of the rescuer and the rescuer (S270).

Further, the control server may change the transmission period based on the rescuer location information and the rescuer location information. For example, the control server may calculate a distance between the rescuer and the rescuer, set a transmission period having a positive correlation with the calculated distance, and transmit the set transmission period to the unmanned aerial vehicle. The unmanned aerial vehicle is

The control server corrects the evacuation route and the rescue route according to the updated location information (S280).

The control server checks whether the acquired location information of the rescuer and the location information of the rescuer have been updated. That is, when the previously received location information of the rescuer and/or the location information of the rescuer is changed by comparing the location information of the rescuer and the location information of the rescuer newly received, it is determined that the location information has been updated.

For example, the control server calculates the distance between the rescuer and the rescuer based on the rescuer location information and the rescuer location information, sets a transmission period having a positive correlation with the calculated distance, and sets the transmission period Is transmitted to the unmanned aerial vehicle.

The unmanned aerial vehicle corrects the evacuation route and the rescue route according to the updated location information of the rescuer and the location information of the rescuer.

In another modification, the control server may set the location information update period and/or the evacuation route and the correction period of the rescue route in an area with a large number of cases of the rescuer's movement route to be shorter than otherwise. For example, the location information update period (and/or the correction period of the route) when a rescuer is located on several roads can be set to be shorter than when the rescuer is placed on a dead end. The control server transmits the set transmission period to the unmanned aerial vehicle, and the unmanned aerial vehicle corrects the evacuation route and the rescue route according to the updated location information of the rescuer and the location information of the rescuer.

7 is a flowchart illustrating a method of checking whether a disaster occurs in an unmanned aerial vehicle according to an embodiment of the present invention.

Referring to FIG. 7, first, it is determined whether or not a disaster in the flight area has occurred (S222).

The unmanned aerial vehicle can be determined by receiving information on the occurrence of a disaster from a control server or a remote control device. The information on the occurrence of a disaster may include any one or more of a signal indicating the occurrence of the disaster itself and a range of the disaster occurrence. Alternatively, the unmanned aerial vehicle may receive map information on a disaster occurrence area from a control server or a remote control device and compare it with the flight location to determine whether a disaster occurs in the flight area.

Next, the unmanned aerial vehicle determines whether there is an alarm device that transmits a disaster broadcast to the flight area (S234).

The unmanned aerial vehicle determines whether an alarm device is located within the flight path. The flight movement path and alarm device may be received from the control server or may be input from a storage device such as USB before flight. In addition, the broadcast transmission range of the alarm device can be received from the control server or input from a storage device such as USB before flight. For this purpose, the control unit may monitor in real time or periodically whether the flight position is within the transmission range of the pre-stored alarm device.

The unmanned aerial vehicle determines whether a disaster broadcast is normally transmitted from the alarm device (S236).

The unmanned aerial vehicle analyzes whether a disaster alert broadcast is included in the audio detected through the microphone. The unmanned aerial vehicle 100 may directly analyze whether the disaster warning broadcast is included by comparing the disaster warning broadcast sample stored in advance in the storage unit with the audio signal detected through the microphone. In addition, by transmitting the audio detected through the microphone to the control server 300, the control server 300 may be implemented to receive a result of determining whether or not the alarm device is normally transmitted.

The unmanned aerial vehicle determines that it is not necessary to transmit a disaster warning broadcast when the audio detected through the microphone includes a disaster warning broadcast. On the other hand, if a disaster warning broadcast is included in the audio detected through the microphone, it is determined that it is necessary to transmit the disaster warning broadcast.

The above-described method can be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention includes one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs). , Field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor through various known means.

In the above, embodiments disclosed in the present specification have been described with reference to the accompanying drawings. As described above, the embodiments shown in each drawing are not to be construed as limiting, and may be combined with each other by those skilled in the art who are familiar with the contents of the present specification, and when combined, some elements may be interpreted as being omitted.

Here, the terms or words used in the specification and claims should not be construed as being limited to a conventional or dictionary meaning, but should be interpreted as meanings and concepts consistent with the technical idea disclosed in the present specification.

Accordingly, the embodiments described in the present specification and the configurations shown in the drawings are only one embodiment disclosed in the present specification, and do not represent all the technical ideas disclosed in the present specification, and thus various alternatives that can be substituted for them at the time of application It should be understood that there may be equivalents and variations.

100: unmanned aerial vehicle
110: wireless transceiver
120: control unit
130: position receiver
140: video recording unit
150: message generator
160: alarm output unit
180: power supply
190: data storage unit
200: remote control device
300: control server
400: alarm device
500: rescuer terminal

Claims (10)

While flying in a certain area, by analyzing the image information acquired from one or more cameras, it determines whether the flight area is a disaster area and whether a rescuer exists, and if it is determined that it is a disaster area and a rescuer exists, the location of the rescuer is detected. Unmanned aerial vehicle;
A rescuer terminal for transmitting its own real-time location information; And
Receiving image information and location information of the rescuer and real-time location information of the rescuer terminal, set a rescue route and an evacuation route of the rescuer, provide a rescue route to the rescuer terminal, and provide an evacuation route to the unmanned aerial vehicle. Including a control server to provide;
The control server shortens the location information update period and the correction period of the evacuation route and rescue route as the rescuer and the rescuer's locations are closer, and the location information update period and the evacuation route and rescue route as the number of rescue routes increases. Shorten the correction cycle of
The control server calculates the distance between the rescuer and the rescuer based on the rescuer location information and the rescuer location information, sets a transmission period having a positive correlation with the calculated distance, and sets the set transmission period to the unattended Transfer to the vehicle,
The unmanned aerial vehicle is an information processing system for the structure of a mountainous area using an unmanned aerial vehicle that transmits image information and location information to the control server according to the transmission period. delete The method of claim 1,
The control server is an information processing system for the rescue of a mountainous area using an unmanned aerial vehicle that transmits disaster evacuation information including any one or more of a disaster occurrence alarm and an evacuation method based on the location and image information of the rescuer to the unmanned aerial vehicle . The method of claim 1,
The unmanned aerial vehicle is an information processing system for the rescue of a mountainous area using an unmanned aerial vehicle that transmits a message including disaster evacuation information received from the control server through a speaker. The method of claim 1,
The unmanned aerial vehicle includes a microphone and a speaker,
The unmanned aerial vehicle determines whether it is necessary to transmit an alert broadcast based on the sound information and the image information acquired from the microphone, and accordingly processes information for the rescue of a mountainous area using an unmanned aerial vehicle that transmits a disaster alert broadcast through a speaker. system. Acquiring image information from one or more cameras while flying the unmanned aerial vehicle in an arbitrary mountain area, analyzing the acquired image information, and determining whether a disaster has occurred in the flight area and whether a rescuer exists;
If the unmanned aerial vehicle determines that a rescuer exists, detecting the location of the rescuer, and transmitting the detected location information and image information of the detected location to at least one of a rescuer terminal and a control server registered in advance;
Receiving, by a control server, image information and location information of the rescuer and real-time location information of the rescuer terminal, and setting a rescue route and an evacuation route of the rescuer according to a previously stored rescue scenario;
The control server providing a rescue route to the rescuer terminal and providing an evacuation route to the unmanned aerial vehicle; And
Including the step of the unmanned aerial vehicle guided flight along the evacuation route,
The control server shortens the location information update period and the correction period of the evacuation route and rescue route as the rescuer and the rescuer's locations are closer, and the location information update period and the evacuation route and rescue route as the number of rescue routes increases. Shorten the correction cycle of
The control server calculates the distance between the rescuer and the rescuer based on the rescuer location information and the rescuer location information, sets a transmission period having a positive correlation with the calculated distance, and sets the set transmission period to the unattended A method of an information processing system for the rescue of a mountainous area using an unmanned aerial vehicle transmitted to a vehicle. delete The method of claim 6,
An information processing system for the rescue of a mountainous area using an unmanned aerial vehicle in which the control server transmits, to the unmanned aerial vehicle, disaster evacuation information including any one or more of a disaster occurrence warning and evacuation instructions based on the location and image information of the rescuer Way. The method of claim 6,
The unmanned aerial vehicle is an information processing system for the rescue of a mountainous area using an unmanned aerial vehicle that transmits a message including disaster evacuation information received from the control server through a speaker. The method of claim 6,
Dividing the mountain area into a plurality of areas by the control server and setting a disaster urgency level for each divided area; And
The method of the information processing system for the rescue of a mountainous area using an unmanned aerial vehicle further comprising the step of the control server setting different disaster warning information according to the urgency level and providing it to an alarm device of a matching location.

Images