KR102045984B1 - System and method for wireless image transmission of unmanned aerial vehicle - Google Patents

Abstract

The present invention relates to a wireless image transmission system. The wireless image transmission system comprises a high-speed nonvolatile memory in which data is recorded by a first control part; a second control part for reading the data recorded in the high-speed nonvolatile memory; a bidirectional asynchronous data exchange device. The first control part and the second control part write and read the data in the high speed nonvolatile memory through the bidirectional asynchronous data exchange device. The bidirectional asynchronous data exchange device may control the second control part not to read the data from the high-speed nonvolatile memory while the data is written to the high-speed nonvolatile memory. It is possible to transmit metadata wirelessly in real time.

Description

System and method for wireless image transmission of unmanned aerial vehicle

The present invention relates to a wireless image transmission system and method for an unmanned aerial vehicle (UAV), and to a system and method for wireless transmission of high resolution image / photo or metadata captured in an unmanned aerial vehicle in real time.

Recently, unmanned aerial vehicles (UAVs) are often equipped with cameras to take high-resolution photos or videos during flight.These image information is stored in a storage device such as a micro SD (μSD) card. The unmanned aerial vehicle controls the user's computer by directly removing the micro SD card in order to transmit the video information stored in the micro SD (μSD) or other storage device installed in the drone to the cloud. There is an inconvenience to connect to.

In addition, there is a problem in that image information stored in μSD cannot be obtained when the drone is not safely returned after the flight is terminated due to an unexpected situation or abnormal operation.

On the other hand, there is a case of streaming wireless transmission of video information photographed from an unmanned aerial vehicle. For such streaming wireless transmission, the captured video information is lost and compressed so that it is included in not only high resolution video / photo information but also video / photo necessary for the system. There is a problem in that various metadata such as GPS-based location information, shooting environment information of the aircraft and camera, and thermal image may not be obtained.

In addition, the drone is capable of automatic flight and a system for managing a plurality of drones is being automated, but the μSD card has to be removed from the drone and inserted into a computer. There is a problem in that it is not possible to build various types of automated service platforms such as transmitting the collected image information to the platform for automatic analysis and informing the customer.

Korean Patent No. 10-1840922 discloses a video or recorded image taken by an unmanned aerial vehicle in real time in a database of an unmanned aerial vehicle operator, a telecommunication company or a control center, and stored in a cloud or computer of a user. The lossy compression of the captured video results in various information such as GPS-based location information included in the video / pictures necessary for system automation, environment information of the aircraft and camera, and temperature information in the case of thermal imaging. The problem is that metadata cannot be transmitted.

Korean Patent Publication No. 10-1670767 uses a redundant transmission streaming in order to transmit video information without interruption of the video in real time by mounting video equipment on an unmanned aerial vehicle. Not only high resolution video / photo information but also various metadata such as GPS based location information included in video / photo necessary for system automation, environment information of aircraft and camera, temperature information in case of thermal image, etc. There is a problem that can not be.

Korean Patent Publication No. 10-1981918 has a GCS agent (Ground Control Station Agent) for integrating and operating a plurality of unmanned aerial vehicles in consideration of the ever-changing drone operating environment, the GCS agent is a flight unmanned aerial vehicle Although the captured image is received and transmitted back to the GCS (Ground Control Station), such a general image transmission loses and compresses the captured image, so that not only high resolution image / photo information but also image / photo necessary for automation of the system There is a problem in that various metadata such as GPS-based location information, shooting environment information of the aircraft and camera, and thermal image may not be transmitted.

Patent Registration No. 10-1840922 Patent Registration No. 10-1670767 Patent Publication No. 10-1981918

An object of the present invention is to transmit a high-resolution image / photo taken from the drone and the metadata contained therein in real time wirelessly.

In addition, another object of the present invention is to transmit a high resolution image / photo taken from the unmanned aerial vehicle and the metadata contained therein in flight to ensure secure.

Still another object of the present invention is to be able to automate a system for providing a service using an unmanned aerial vehicle.

In addition, another object of the present invention is to enable real-time checking of high-resolution images / pictures on an automated workflow to perform tasks such as quick remote inspection and decision making.

Yet another object of the present invention is to enable additional unmanned aerial vehicle mission settings and modifications through rapid remote inspection.

The problem to be solved by the present invention is not limited only to the above purpose, other technical problems not explicitly indicated above are easily understood by those skilled in the art through the configuration and operation of the present invention. Could be.

In this invention, the following structures are included in order to solve the said subject.

The present invention relates to a wireless image transmission system, comprising: a high speed nonvolatile memory in which data is recorded by a first control unit; A second control unit for reading the data recorded in the fast nonvolatile memory; And a bidirectional asynchronous data exchange device, wherein the first control unit and the second control unit write and read data to and from the high speed nonvolatile memory through the bidirectional asynchronous data exchange device, and the bidirectional asynchronous data exchange device includes the bidirectional asynchronous data exchange device. And while the data is recorded in the volatile memory, the second control unit controls such that the data cannot be read from the high speed nonvolatile memory.

The bidirectional asynchronous data exchange device of the present invention is connected to the first control unit via a SD (Secure Digital) card interface controller.

The first control unit of the present invention is characterized by controlling a camera mounted on an unmanned aircraft or an unmanned aerial vehicle.

The bidirectional asynchronous data exchange device of the present invention includes a memory monitor for detecting whether data is written to the fast nonvolatile memory by a first controller, and data written to the fast nonvolatile memory by a first controller or by a second controller. And a bus switcher for connecting data to be read from the high speed nonvolatile memory, wherein the memory monitor is configured to connect the bus switcher so that data is preferentially written to the high speed nonvolatile memory by a first controller, and to a first controller. The bus switcher may be connected to read data from the fast nonvolatile memory by a second controller when data is not written to the fast nonvolatile memory.

The high speed nonvolatile memory of the present invention is characterized by being a NAND flash or a NOR flash capable of writing or reading at high speed by parallel processing.

The present invention also relates to a wireless image transmission method, comprising: writing data by a first control unit into a high speed nonvolatile memory through a bidirectional asynchronous data exchange device; Reading out the data from the high speed nonvolatile memory through a bidirectional asynchronous data exchange device by a second control unit; Transmitting the data read from the fast nonvolatile memory by a wireless transmitter; Wherein the bidirectional asynchronous data exchange device controls the second controller not to read data from the high speed nonvolatile memory while data is written to the high speed nonvolatile memory.

The present invention may also be a computer program stored in a medium for executing the wireless image transmission method.

The present invention has the effect of enabling the real-time wireless transmission of high-resolution images / photos, metadata taken by the drone.

In addition, the present invention has the effect that it is possible to securely secure a high-resolution image / photo, metadata taken by the drone.

The present invention also has the effect of being able to automate a system for managing a large number of unmanned aerial vehicles.

In addition, the present invention has the effect of enabling a task such as fast remote inspection and decision making on an automated workflow.

In addition, the present invention has the effect of enabling additional unmanned aerial vehicle mission settings and changes through a quick remote inspection.

Effects by the present invention is not limited only to the above effects, and other effects not explicitly indicated above will be readily understood by those skilled in the art through the construction and operation of the present invention.

1 illustrates an embodiment of a wireless video transmission system of the present invention.
2 illustrates another embodiment of a wireless video transmission system of the present invention.
Figure 3 shows the internal configuration of the bidirectional asynchronous data exchange device of the present invention.
4 shows an internal circuit diagram of the bus switcher of the present invention.
5 (a) and 5 (b) illustrate an operation of storing the captured image in the wireless image transmission system of the present invention.
6 (a) and 6 (b) illustrate an operation of reading a stored image in the wireless image transmission system of the present invention.
7 is a flowchart illustrating a wireless image transmission method of the present invention.

Hereinafter will be described the overall configuration and operation according to a preferred embodiment of the present invention. These examples are illustrative and do not limit the configuration and operation of the present invention, other configurations and functions not explicitly shown in the embodiment through the embodiments of the present invention to the common knowledge in the art to which the present invention belongs. If the person can easily understand it will be seen as a technical idea of the present invention.

Recently, drones, unmanned aerial vehicles (UAVs), are used in various industries by surveying dangerous areas or performing aerial surveys efficiently on behalf of humans. Although high-resolution video / photo data and metadata contained therein for surveying operations are typically stored on a micro Secure Digital (SD) card, when a drone completes its mission and returns, a person may enter the micro SD. (μSD) card is inserted into the computer and examined.

The drone is capable of automatic flight and the system for managing a large number of drones is being automated, but since the μSD card has to be manually removed from the drone and inserted into a computer, the present invention does not use the μSD card. It is possible to wirelessly transmit high resolution video / photo data and metadata taken by an unmanned aerial vehicle, and apply it to an existing drone using a μSD card as it is so that it can wirelessly transmit high resolution video / photo and metadata taken by an unmanned aerial vehicle. Doing.

1 illustrates an embodiment of a wireless video transmission system of the present invention.

Referring to FIG. 1, the wireless image transmission system 300 of the present invention includes a card interface (I / F) controller 3120, a bidirectional asynchronous data exchange device 320, a central processing unit (CPU) 330, and a high speed ratio. It includes a volatile memory 340, communication modem 350, USB interface (I / F) 360, and is connected to the microSD card slot of the drone 100 or camera 200 via a microSD extension cable, communication modem The 350 and the USB interface (I / F) 360 are respectively connected to a cloud system and a personal computer (PC).

The card I / F controller 310 is connected to the microSD card slot of the drone 100 or the camera 200 through a microSD extension cable, and inputs the image / photo data and metadata input through the microSD card slot. It is transmitted to the exchange device (320).

In addition, the drone 100 has a camera which is basically mounted on the drone 100 and the embodiment in which the camera 200 in addition to the drone 100 is shown, as another embodiment, μSD card slot is formed in the camera itself Although it may be, it is shown that the drone 100 may be formed in the body portion.

Although not shown in the drawing, the CPU (Central Processing Unit) of the drone 100 or the camera 200 transmits image / photo data and metadata through the μSD card slot, and the data is the bidirectional asynchronous data. It is transmitted to and stored in the high speed nonvolatile memory 340 via the exchange device 320.

The bidirectional asynchronous data exchange device 320 transmits image / photo data and metadata input from the drone 100 or the camera 200 to the high speed nonvolatile memory 340, and transmits the data to the high speed nonvolatile memory 340. When the data is stored, the central processing unit (CPU) 330 of the wireless image transmission system 300 may read the stored data.

When image / photo data and metadata are stored in a conventional μSD card which is detached to the μSD card slot, the CPU of the drone 100 or the camera 200 and the μSD card operate in a master-slave mode and the drone 100 If the image is taken in the μSD card is not accessible from the external CPU (330).

However, in the present invention, even when the image is taken by the drone 100 or the camera 200 by using the bidirectional asynchronous data exchange device 320, the data access / of the CPU 330 to the fast nonvolatile memory 340 / The reading module can be accessed, allowing the recording and reading of images in parallel.

That is, a delay time for recording the photographed image may occur even while the drone 100 or the camera 200 is capturing the image. When the delay time is detected, the central processing unit (CPU) 330 is bidirectional. The data stored in the high speed nonvolatile memory 340 may be read through the asynchronous data exchange device 320.

The data transmission module of the CPU 330 may transmit the read image / photo data and metadata to the communication modem 350 or the USB I / F 360, and the communication modem 350 may transmit the high speed nonvolatile data. The data read from the memory 340 may be wirelessly transmitted, and the communication modem 350 may read data transmitted by the CPU 330 through a data rate transmitted through a wireless network such as 4G, 5G, or Wi-Fi. Since it is generally smaller than the data rate of the data, it is required to be controlled by the flow control module of the CPU 330, and the file capacity of the image / photo data and metadata in which the image is taken and stored in the drone 100 or the camera 200. And the flow control in consideration of the storage capacity of the fast nonvolatile memory 340.

The high speed nonvolatile memory 340 may be a NAND flash or a NOR flash, and the writing speed of the high speed nonvolatile memory 340 is equal to or higher than a data rate of an image transmitted from the bidirectional asynchronous data exchange device 320. It is preferable to have a writing speed of at least 400 MB / s or more, and to increase the writing and reading speed by parallel processing such as a solid state drive controller (SSD) rather than a general flash memory. .

According to the present invention, the card I / F controller 310 and the microSD card slot are connected to each other through a microSD extension cable, so that the present invention can be applied to an existing drone using a microSD card, and the high-resolution image / photo data and metadata photographed by the drone. To enable wireless transmission. In other words, by connecting the microSD extension cable to the microSD card slot instead of the microSD card, it is possible to wirelessly transmit high-definition video / photo data and metadata captured by the drone so that a person does not need to remove the microSD card and manage multiple drones. The system can also be automated.

In addition, even if the drone falls due to unexpected situation and abnormal operation, and can not be safely returned after the flight, the high resolution image taken by the drone can be transmitted wirelessly in real time to secure the data stored in μSD.

Meanwhile, although streaming wireless transmission of image information photographed by a drone has been conventionally performed, lossy compression of the captured image information is not possible to secure high resolution image information for such streaming wireless transmission. By transmitting the image / picture stored in the high speed nonvolatile memory 340 in units of files, high resolution image / photo information and metadata may be secured.

In addition, according to the present invention, according to the wireless transmission of images / pictures taken by the drone in real time, a faster remote inspection is possible in the automated workflow receiving the images / pictures, and thus the performance of tasks such as decision making is also performed. It can be done quickly. If a quick remote check is made, additional drone missions can be set and changed later.

In addition, the present invention may transmit data to the cloud system by the communication modem 350, but may be connected to a personal computer (PC) with a USB interface (I / F) 360 to transmit data. That is, when the drone lands, the manager may connect the USB terminal of the notebook PC with the USB interface (I / F) 360 of the present invention and then receive data.

2 illustrates another embodiment of a wireless video transmission system of the present invention.

Referring to FIG. 2, the wireless image transmission system 300 of the present invention includes a bidirectional asynchronous data exchange device 320, a central processing unit (CPU) 330, a high speed nonvolatile memory 340, a communication modem 350, A USB interface (I / F) 360, and the image / photo data and metadata captured by the drone 100 or the camera 200 to the bidirectional asynchronous data exchange device 320 without going through a microSD card slot. Direct transmission, the communication modem 350 and the USB interface (I / F) 360 is connected to a cloud system and a personal computer (PC).

In another embodiment of the present invention shown in FIG. 2, unlike the embodiment of the present invention described in FIG. 1, the image / photo data and metadata photographed by the drone 100 or the camera 200 are bidirectional. Direct transmission to the asynchronous data exchange device 320 is connected to the drone 100 or the camera 200 without having a microSD card slot and a card interface (I / F) controller 310.

One embodiment of the present invention described in FIG. 1 is generally compatible with drones that store image / photo data and metadata in a μSD card, thereby improving compatibility, but storing data at a drone-producing manufacturer 2 is illustrated in FIG. 2 because it can be configured to directly transmit a captured image / photo to the wireless image transmission system 300 without having a μSD card slot.

Figure 3 shows the internal configuration of the bidirectional asynchronous data exchange device of the present invention.

Referring to FIG. 3, the bidirectional asynchronous data exchange device 320 includes a memory monitor 321 and a bus switcher 322, wherein the memory monitor 321 is a high-speed ratio of data input from the μSD interface controller 310. The bus switcher 322 is connected to be transferred to the NAND flash 340 which is a high speed nonvolatile memory when data is input from the drone 100 or the camera 200. When data is not input from the drone 100 or the camera 200, the CPU 330 is connected to read data from the NAND flash 340 which is a high speed nonvolatile memory.

The memory monitor 321 and the bus switcher 322 preferentially receive data from the μSD interface controller 310 and transmit the data to the NAND flash 340 which is a high speed nonvolatile memory. ) May read data stored in the NAND flash 340 when data is not transmitted from the μSD interface controller 310 to the NAND flash 340 which is a high speed nonvolatile memory.

The bidirectional asynchronous data exchange device 320 including the memory monitor 321 and the bus switcher 322 is a Field Programmable Gate Array (FPGA) or Complex Programmable Logic Device (CPLD). And so on.

4 shows an internal circuit diagram of the bus switcher of the present invention.

Referring to FIG. 4, when the memory monitor 321 detects that data is input from the drone 100 or the camera 200, the data bus from the card I / F controller and the NAND flash 340 which is a high speed nonvolatile memory. The switch is turned on to connect the data bus connected to the data bus, and the data bus connected to the NAND flash 340 which is a high-speed nonvolatile memory and the data bus connected to the CPU 330 are provided with an inverter so as to prevent the connection. It is preferable.

On the contrary, when the memory monitor 321 detects that data is not input from the drone 100 or the camera 200, the data bus from the card I / F controller is connected to the NAND flash 340, which is a high speed nonvolatile memory. The data bus is switched off so that the connection is prevented, and the data bus connected to the NAND flash 340 which is a high speed nonvolatile memory and the data bus connected to the CPU 330 are connected. The data bus can also consist of any n (n is integer) bits, such as 16 bits, 32 bits, or 64 bits.

5 (a) and 5 (b) illustrate an operation of storing the captured image in the wireless image transmission system of the present invention.

Referring to FIGS. 5A and 5B, the bidirectional asynchronous data exchange device 320 includes a memory monitor 321 and a bus switcher 322, and the memory monitor 321 includes a μSD interface controller ( Sensing that data input from 310 is transmitted to and written to NAND flash 340, and when data is input from the μSD interface controller 310, the switch is turned on via bus switcher 322 to provide high-speed nonvolatile It is connected to be transferred to the NAND flash 340 which is a memory.

The memory monitor 321 and the bus switcher 322 preferentially input data from the μSD interface controller 310 to be transferred to and stored in the NAND flash 340 which is a high speed nonvolatile memory. The switch to the CPU 330 is turned off so that data stored in the NAND flash 340 cannot be read.

6 (a) and 6 (b) illustrate an operation of reading a stored image in the wireless image transmission system of the present invention.

6 (a) and 6 (b), the bidirectional asynchronous data exchange device 320 includes a memory monitor 321 and a bus switcher 322, the memory monitor 321 is a microSD interface controller ( Detects whether data input from 310 is transmitted to the NAND flash 340 and is recorded. When no data is input from the μSD interface controller 310, the bus switcher 322 is switched off to turn off the μSD interface. The data bus from the controller 310 is prevented from being connected to the NAND flash 340, and the switch from the NAND flash 340 to the CPU 330 is turned on so that the CPU 330 is connected to the NAND flash 340. The data stored in) can be read.

7 is a flowchart illustrating a wireless image transmission method of the present invention.

Referring to FIG. 7, the wireless image transmission method of the present invention includes the steps of writing data to a high speed nonvolatile memory by a first control unit through a bidirectional asynchronous data exchange device; Reading out the data from the high speed nonvolatile memory through a bidirectional asynchronous data exchange device by a second control unit; And transmitting the data read from the high speed nonvolatile memory by a wireless transmitter, wherein the bidirectional asynchronous data exchange device includes a second control unit configured to transmit the data while the data is written to the high speed nonvolatile memory. Control is performed so that data cannot be read from the memory.

In the step (S100) in which data is written to the high speed nonvolatile memory 340 by the first control unit via the bidirectional asynchronous data exchange device 320, the high speed nonvolatile memory (the CPU of the drone 100 or the camera 200) In the case where the data is recorded in the 340 and the card interface (I / F) controller 310 is provided, the data may be recorded in the high speed nonvolatile memory 340 from the μSD card slot through the μSD extension cable.

In the step S200 of reading the data from the high speed nonvolatile memory 340 through the bidirectional asynchronous data exchange device 320 by the second control unit, the high speed nonvolatile memory ( Data stored in the 340 may be read, and the bidirectional asynchronous data exchange device 320 may include the CPU 330 in the high speed nonvolatile memory 340 even when the drone 100 or the camera 200 captures an image. The data access / read module of the module can be accessed to simultaneously record and read images.

That is, a delay time for recording the photographed image may occur even while the drone 100 or the camera 200 is capturing the image. When the delay time is detected, the CPU 330 may detect the delay time. The high speed nonvolatile memory 340 may be accessed through the bidirectional asynchronous data exchange device 320 to read stored data.

In the step S300 of transmitting the data read from the high speed nonvolatile memory 340 by the wireless transmitter 350, the wireless data may be wirelessly transmitted from the high speed nonvolatile memory 340. The denial communication modem 350 controls the flow of the CPU 330 because the data rate transmitted through the wireless network such as 4G, 5G, Wi-Fi is generally smaller than the data rate of the image read by the CPU 330. Control by a module is required, and flow control may be performed in consideration of the file capacity of data to be taken and stored in the drone 100 or the camera 200 and the storage capacity of the fast nonvolatile memory 340. .

The high speed nonvolatile memory 340 may be a NAND flash or a NOR flash, and the writing speed of the high speed nonvolatile memory 340 is equal to or higher than a data rate of data transmitted from the bidirectional asynchronous data exchange device 320. It is preferable to have a writing speed of at least 400 MB / s or more, and to increase the writing and reading speed by parallel processing such as a solid state drive controller (SSD) rather than a general flash memory. .

In addition, the flowchart of the wireless image transmission method may be implemented by a computer program, and each component of the present invention may be implemented by hardware or software. In addition, the card interface controller 310 of the present invention may be specifically implemented as the μSD interface controller 310, and may be modified to be designed to be connected to a memory card of another standard in addition to the μSD card.

100: drone
200: camera
300: wireless video transmission system
310: card I / F controller
320: bidirectional asynchronous data exchange
330: CPU
340: fast nonvolatile memory
350: communication modem
360: USB I / F

Claims (7)

In the wireless video transmission system of the unmanned aerial vehicle,
A high speed nonvolatile memory in which photographing data is recorded by the first control unit;
A second control unit which reads out the photographing data recorded in the fast nonvolatile memory;
A bidirectional asynchronous data exchange device;
The first control unit and the second control unit records and reads recording data into the high speed nonvolatile memory through the bidirectional asynchronous data exchange device.
The bi-directional asynchronous data exchange device is a memory monitor for detecting whether photographing data is recorded in the fast nonvolatile memory by a first control unit,
A bus switcher for connecting the recording data to the high speed nonvolatile memory by the first control unit or the reading data to be read from the high speed nonvolatile memory by the second control unit;
The memory monitor connects the bus switcher so that shooting data is preferentially recorded in the high speed nonvolatile memory by a first control unit, and second when the shooting data is not recorded in the high speed nonvolatile memory by the first control unit. Connecting the bus switcher so that control data is read by the controller from the high speed nonvolatile memory;
And the second controller performs flow control in consideration of the data rate transmitted through the wireless network and the storage capacity of the fast nonvolatile memory.

The method of claim 1,
The two-way asynchronous data exchange device is connected to the first control unit via a SD (Secure Digital) card interface controller, wireless image transmission system of the unmanned aerial vehicle.

The method of claim 1,
The first control unit is a wireless image transmission system of an unmanned aerial vehicle, characterized in that for controlling the camera mounted on the unmanned aerial vehicle or unmanned aerial vehicle.

delete The method of claim 1,
And the high speed nonvolatile memory is a NAND flash or a NOR flash capable of writing or reading at high speed by parallel processing.

In the wireless video transmission method of the unmanned aerial vehicle,
Recording, by the first control unit, the photographing data into the high speed nonvolatile memory through the bidirectional asynchronous data exchange apparatus;
Reading out the photographing data from the fast nonvolatile memory by a second control unit through a bidirectional asynchronous data exchange device;
Transmitting the photographing data read from the fast nonvolatile memory by a wireless transmission unit;
Including,
The bi-directional asynchronous data exchange device is a memory monitor for detecting whether photographing data is recorded in the fast nonvolatile memory by a first control unit,
A bus switcher for connecting the recording data to the high speed nonvolatile memory by the first control unit or the reading data to be read from the high speed nonvolatile memory by the second control unit;
The memory monitor connects the bus switcher so that shooting data is preferentially recorded in the high speed nonvolatile memory by a first control unit, and second when the shooting data is not recorded in the high speed nonvolatile memory by the first control unit. Connecting the bus switcher so that control data is read by the controller from the high speed nonvolatile memory;
And a second control unit performs flow control in consideration of the data rate transmitted through the wireless network and the storage capacity of the fast nonvolatile memory.

A computer program stored in a medium for executing the wireless video transmission method of claim 6 unmanned aerial vehicle.

Images