KR20200082466A - Method for processing and tracking video data from EO-IR camera installed in unmanned aerial vehicle and system thereof - Google Patents

Abstract

According to one embodiment of the present invention, the method of processing a video of an EO-IR camera installed in an unmanned aerial vehicle comprises: a step wherein a video data reception unit receives an original video data outputted from the camera for a preset period and the frame numbers of frames composing the original video data; a step wherein a video buffer storage unit separates the received original video data for each frame, and respectively stores the data in frame buffers placed for each frame; a step wherein a number buffer storage unit stores the received frame numbers in a number buffer; a step wherein a video compression unit receives the original video data stored in the frame buffer and comprehensively compresses the data; and a step wherein a synchronization processing unit synchronizes the frame numbers stored in the number buffer for each frame of the compressed video. The present invention aims to provide the method of processing the video of the EO-IR camera installed in the unmanned aerial vehicle, capable of minimizing the difference between the point of time for showing the video on ground equipment and the point of time for photographing by the EO-IR equipment on the unmanned aerial vehicle.

Description

Electro-optical-infrared camera image processing method, image tracking method and its system {Method for processing and tracking video data from EO-IR camera installed in unmanned aerial vehicle and system thereof}

The present invention relates to an image processing method, an image tracking method, and a system thereof of an electro-optical-infrared camera mounted on an unmanned aerial vehicle, and more specifically, an electro-optical-infrared camera mounted on an unmanned aerial vehicle communicating with ground equipment. A method for processing or tracking an image and a system for implementing the method.

Electro-optical-infrared (EO-IR) equipment mounted on unmanned aerial vehicles transmits and receives various commands and video data while communicating with ground equipment. At this time, a delay in data transmission may occur depending on a data transmission/reception path, a state of a wireless network, and ground equipment.

The occurrence of the delay in data transmission as described above inevitably causes a difference between the time when the image received on the ground equipment is displayed and the time when the EO-IR equipment mounted on the unmanned aerial vehicle photographs the image. In particular, when the speed of movement of a target photographed with EO-IR equipment is excessively fast or shows a movement exceeding a certain level, the image taken by the unmanned aerial vehicle reaches the ground equipment and corresponds to the ground equipment. In the course of the command to capture the target from the aircraft to the unmanned aerial vehicle, the target was deviated from the point where it was originally captured, and eventually the unmanned aerial vehicle's EO-IR equipment continued to capture the target. The tracking accuracy decreases, which is difficult to maintain.

1. Republic of Korea Registered Patent Publication No. 10-1837407 (Publication of March 12, 2018) 2. Republic of Korea Patent Publication No. 10-2011-0113948 (released on October 19, 2011) 3. Republic of Korea Registered Patent Publication No. 10-1394164 (published on May 16, 2014)

The technical problem to be solved by the present invention is to minimize the difference between the image viewing time on the ground equipment and the shooting time of the EO-IR equipment of the unmanned aerial vehicle, the image processing method of the electronic optical-infrared camera mounted on the unmanned aerial vehicle, and the image tracking method And providing the system.

The method according to an embodiment of the present invention for solving the above technical problem is an image processing method of an electro-optical-infrared (EO-IR) camera mounted in an unmanned aerial vehicle, wherein the image data receiving unit outputs the original image from the camera An image data receiving step of receiving frame numbers of data and frames constituting the raw image data; An image buffer storage step in which the image buffer storage unit separates the received raw image data for each frame, and stores each of the received raw image data in a frame buffer provided per frame; A number buffer storage step in which the number buffer storage unit stores the received frame number in a number buffer; An image compression step in which the image compression unit receives the raw image data stored in the frame buffer and collectively compresses it; And a synchronization processing step in which the synchronization processing unit synchronizes the frame numbers stored in the number buffer for each frame of the compressed image.

The method according to another embodiment of the present invention for solving the above technical problem is a method for tracking an image of an electro-infrared camera mounted on an unmanned aerial vehicle, and includes a first frame number, gate size, and gate coordinates from ground equipment. Tracking information receiving step of receiving the image tracking information; A frame search step of searching for a video frame for the first frame number in an image buffer if the received first frame number is a number of a frame at a point earlier than the second frame number for the current time; An algorithm selection step of selecting any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number; And the image size tracking the movement of the target with respect to the gate size and the gate coordinates from the video frame of the first frame number to the video frame of the second frame number according to the tracking speed determined by the selected tracking algorithm. Steps.

The system according to another embodiment of the present invention for solving the above technical problem is an image processing system of an electro-optical-infrared (EO-IR) camera mounted in an unmanned aerial vehicle, the raw image data output from the camera and An image data receiving unit receiving frame numbers of frames constituting the raw image data; an image buffer storage unit separating the received raw image data into frames, and storing each of the received raw image data in a frame buffer provided for each frame; A number buffer storage unit for storing the received frame number in the number buffer; An image compression unit for receiving the raw image data stored in the frame buffer and compressing it in a batch; And a synchronization processing unit for synchronizing the frame number stored in the number buffer for each frame of the compressed image.

The system according to another embodiment of the present invention for solving the above technical problem is an image processing system of an electro-optical-infrared camera mounted on an unmanned aerial vehicle, wherein a first frame number, gate size, and gate coordinates are received from ground equipment. Tracking information receiving unit for receiving the image tracking information including; A frame search unit for searching the video frame for the first frame number in an image buffer if the received first frame number is a number of a frame at a point earlier than the second frame number for the current time; An algorithm selection unit selecting any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number; And the image size tracking the movement of the target with respect to the gate size and the gate coordinates from the video frame of the first frame number to the video frame of the second frame number according to the tracking speed determined by the selected tracking algorithm. Includes wealth.

One embodiment of the present invention can provide a computer-readable recording medium storing a program for implementing the method.

According to the present invention, it is possible to dramatically improve the initial success rate for the target by minimizing the difference between the image viewing time on the ground equipment and the shooting time of the EO-IR equipment of the unmanned aerial vehicle, and to a remotely controlled platform such as an unmanned aerial vehicle. By increasing the success rate of the installed EO-IR mission, the utilization of the EO-IR equipment can be further improved.

FIG. 1 is a diagram schematically showing an example of a communication process between an EO-IR device of an unmanned aerial vehicle and a ground control device.
2 is a block diagram of an example of an image processing system according to the present invention.
3 is a diagram schematically showing a process in which raw image data and metadata generated from EO-IR equipment are compressed by an image compression system.
4 is a diagram schematically showing an example of an image tracking command transmission flow through the ground control device.
FIG. 5 is a flowchart illustrating a process in which the image compression system compresses the raw image data and transmits it to the ground control device.
FIG. 6 is a flowchart illustrating a process of tracking an image by an image tracking system that receives an image tracking command from a ground control device.

The terminology used in the embodiments has been selected from general terms that are currently widely used as possible while considering functions in the present invention, but this may vary depending on the intention or precedent of a person skilled in the art or the appearance of a new technology. In addition, in certain cases, some terms are arbitrarily selected by the applicant, and in this case, their meanings will be described in detail in the description of the applicable invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the entire contents of the present invention, not a simple term name.

When a certain part of the specification "includes" a certain component, this means that other components may be further included instead of excluding other components unless otherwise specified. In addition, terms such as “…unit” and “…module” described in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram schematically showing an example of a communication process between an EO-IR device of an unmanned aerial vehicle and a ground control device.

Referring to FIG. 1, it can be seen that the image output from the EO-IR equipment 10 of the unmanned aerial vehicle is transmitted to the ground control device 30 through the unmanned aerial vehicle data link module 20. 1 is a schematic diagram for explaining a communication process between the EO-IR equipment and the ground control device of an unmanned aerial vehicle, and it is assumed that the rest of the configuration is omitted except for the configuration for explaining the communication process.

The EO-IR (ElectrOoptic-InfRared) equipment 10 of an unmanned aerial vehicle is largely composed of an ECA module 11, an ICA module 13, an ITA module 15, an IPA module 17, and a CTA module 19.

First, the ECA module 11 is a module including an electro-optical camera, which photographs an object to be photographed through an electro-optical camera, and the ITA module (15) with the raw video data generated by the shooting along with the frame number. ). In this process, in addition to the frame number, various metadata about the raw image data are also transmitted to the ITA module 15.

The ICA module 13 is a module including an infrared camera, which photographs an object to be photographed through an infrared camera, and transmits raw image data generated by the photographing to the ITA module 15 along with the frame number. In this process, in addition to the frame number, various metadata about the raw image data are also transmitted to the ITA module 15.

The ITA module 15 receives the raw image data and metadata about the raw image data from the ECA module 11 and the ICA module 13, and receives the received raw image data and metadata of the raw image data from the IPA module. (17). When 30 frames per second of raw image data and metadata are output from the ECA module 11 and the ICA module 13, the ITA module 15 collects the data and transmits it to the IPA module 17. .

The IPA module 17 receives filtered raw image data and metadata from the ITA module 15, and preprocesses the image compression process for the raw image data, and each of the raw image data through a preset filter. Filter specific data in the frame of. In this process, unnecessary noise may be removed from the raw image data, or a contrast ratio enhancement effect may be applied.

The CTA module 19 separates the raw image data received from the IPA module 17 for each frame. The CTA module 19 may perform data labeling processing so that metadata other than the raw image data can be transferred to a separate buffer of the CTA module 19. The CTA module 19 is a module that performs a process of processing frames and metadata through a series of processes and encoding, and the image compression transmits an image to the ground control device 30 through wireless communication. It is an essential task. The raw image data compressed by the CTA module 19 is transmitted to the ground control device 30 through the unmanned data link module 20.

The raw image data reaching the ground control device 30 is played back after being decompressed, and when the user finds a target to be captured in the raw image data played through the ground control device 30, In other words, it is possible to send a command to the unmanned aerial vehicle to capture the target and keep tracking it.

2 is a block diagram of an example of an image processing system according to the present invention.

Referring to FIG. 2, it can be seen that the image processing system 200 according to the present invention includes an image compression system 210 and an image tracking system 230. The image processing system 200 of FIG. 2 may be included in the EO-IR equipment 10 of the unmanned aerial vehicle described in FIG. 1, or may be physically or logically connected to the EO-IR equipment 10 to operate.

In addition, the image processing system 200 of FIG. 2 and the sub-modules included in the image processing system 200 are the ECA module 11, the ICA module 13, the ITA module 15, and the IPA module described in FIG. (17) and CTA module 19 may be not only a module that processes substantially the same or similar functions, but according to an embodiment, the ECA module 11, the ICA module 13, and the ITA module 15 , Since at least two or more of the IPA module 17 and the CTA module 19 may perform a function, or may include a module performing some functions, a description will be given with reference to FIG. 1.

First, the image compression system 210 is a system that performs a function of transmitting and compressing the compressed image data to the ground control device 30 after receiving and compressing the raw image data photographed by the EO-IR equipment, the image data receiving unit 211, an image buffer storage unit 212, a number buffer storage unit 213, an image compression unit 214, a GPIO signal transmission unit 215, and a synchronization processing unit 216. Depending on the embodiment, the GPIO signal transmission unit 215 may be omitted or may be implemented in a form included in the synchronization processing unit 216.

The image data receiving unit 211 receives the raw image data output from the EO-IR equipment 10 for a predetermined time and frame numbers of frames constituting the raw image data. Here, the image received by the image data receiving unit 211 is a concept including both still images and moving images, and a minimum unit constituting the image is a frame. For example, if the raw image data is 10-frame image data, the raw image data is image data in which ten frames are concatenated in a chronological order, and can be realized as a video through a device capable of continuously reproducing frames. . Each frame of the raw image data received by the image data receiving unit 211 is attached with a frame number, and the frame number is regarded as metadata of the raw image data.

The image buffer storage unit 212 separates the received raw image data for each frame and stores each in a frame buffer provided for each frame. The image buffer storage unit 212 separates the raw image data for each frame, grasps the number of frames, and provides a buffer according to the determined number, and stores each frame for each buffer. Hereinafter, a buffer for storing frames will be referred to as a frame buffer for convenience. Here, the buffer means a memory of a small capacity capable of storing data only for a short time, and has a capacity sufficient to store a frame of a specific resolution according to preset information.

The number buffer storage unit 213 stores the frame number received by the image data receiving unit 211 in a number buffer. More specifically, the number buffer storage unit 213 may accumulate and store frame numbers in the number buffer in a chronological order, and may be stored in a queue method for output and use in a cumulative order.

The image compression unit 214 receives the raw image data stored in the frame buffer and compresses it in a batch. Here, the image compression unit 214 receives the raw image data from the frame buffer and collectively compresses it, and combines the raw image data separated into at least two into one image data without metadata. This means, in this process, it is encoded to transmit image data to the ground control device easily and quickly. For example, the first capacity of the raw image data stored in the 1000 frame buffers has a much larger capacity than the second capacity after being compressed by the image compression unit 214. In the present invention, since a well-known method is used for a method of compressing image data, a method of compressing a plurality of frames received from a frame buffer into one image data is not limited to one specific method.

The GPIO signal transmitter 215 will be described later in FIG. 3 for convenience of description.

The synchronization processing unit 216 synchronizes the frame number stored in the number buffer for each frame of the image compressed by the image compression unit 214. The synchronized compressed image is transmitted to the ground control device 30 together with the frame number.

The ground control device 30 receives and reproduces the synchronized compressed image together with the frame number. The user reviews the video that is reproduced after being decompressed (decoded) in the ground control device 30, captures a tracking target that needs to be tracked in a specific frame (Lock-On), and unattended a tracking command to track the tracking target It can be transmitted to the image processing system 200 of the aircraft. The image processing system 200 may receive a tracking command and transmit the received tracking command to the image tracking system 230 so that a specific target can be continuously tracked by the user.

Referring to FIG. 2, it can be seen that the image tracking system 230 may include a tracking information receiving unit 231, a frame searching unit 233, an algorithm selection unit 235, and an image tracking unit 237. The image tracking system 230 of FIG. 2 analyzes a tracking command received from the ground control device 30, searches for frames of images stored in a video buffer, identifies a tracking target, and proceeds with tracking Perform a function.

The tracking information receiving unit 231 receives image tracking information including the first frame number, gate size, and gate coordinates from the ground control device 30. Here, the first frame number refers to a number for a frame at the moment when the user captures a tracking target from the video data being played through the ground control device 30. For example, in the image data being played from t frame to t+10 frame, the tracking object suddenly appears at the time point at t+4 frame, so that the user commands the ground control device 30 to capture the tracking object. If generated and transmitted to an unmanned aerial vehicle, the first frame number is "t+4".

The gate size (size of gate) means a horizontal and vertical length of a rectangle surrounding a tracking target in a frame corresponding to the first frame number. As an index indicating how large the size of the tracking object is in the frame, it may be expressed in units of pixels. For example, if the overall resolution of the frame is 640*480 pixels, the gate size may be 40*30. The gate size is flexible depending on the size of the frame to be tracked. The coordinate of the gate (coordinate of gate) refers to the coordinates of the center point of the tracking target when the tracking target is specified in the frame corresponding to the first frame number. In the frame for the first frame number, the gate coordinate is an index indicating where the tracking target is specifically, and may be expressed as a 2D coordinate value.

If the first frame number is the number of a frame at a point earlier than the second frame number for the current time, the frame search unit 233 searches the video frame for the first frame number in the image buffer. Here, the image buffer is a storage device included in the image tracking system 230, and stores raw image data photographed by the EO-IR equipment for a specific time in a temporal order. The image buffer can be composed of a non-volatile memory such as a flash memory, as well as a volatile memory that temporarily stores data or resets it at regular intervals in order to secure faster data input/output (I/O) speed. It may be composed of. The first frame information is unique metadata that is generated when the EO-IR device generates raw image data, and the frame search unit 233 matches the first frame information with raw image data stored in the image buffer. It can be used to search for.

As an example, if the first frame number is 1000 and the second frame number for the current time is 1100, the frame search unit 233 considers the first frame number to be the number of the frame at a time before the second frame number. By doing so, it is possible to search the image frame for the first frame number in the image buffer. As another example, if there is no first frame number, or if the first frame number is 1100 and the second frame number is 1100, the frame search unit 233 is considered to have no tracking object received from the ground control device 30. Then, the image frame for the first frame number is not searched in the image buffer. First, the frame search unit 233 can search the video frame corresponding to the first frame number in the image buffer, except when the first frame number does not exist or the first frame number matches the second frame number. do.

The algorithm selection unit 235 may select any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number. For example, if the first frame number is 1000 and the second frame number is 1100, the difference between the first frame number and the second frame number is 100, and the algorithm selection unit 235 is based on the difference between the frame numbers of 100. Then, any one of the tracking algorithms previously stored is selected. As another example, if the number of the first frame is 1000 and the number of the second frame is 10000, the difference between the first frame number and the second frame number is 9000, and the algorithm selection unit 235 is the difference between the frame numbers of 9000. On the basis of, it is possible to select any one of the tracking algorithms previously stored. In the above-mentioned two examples, when the difference between the frame numbers is 100, it means that the tracking object is stamped on the frame taken some time ago from the current time, and when the difference between the frame numbers is 9000, the above-described example Compared to this, it means that the object to be traced is stamped on a frame photographed from an old point of time.

Here, the tracking algorithm is a broad concept including a tracking step. That is, the algorithm selection unit 235, according to the state of the image of the first frame number or the second frame number (target brightness, background density, etc.), according to any one of the various tracking algorithms, the algorithm is determined, Within the determined algorithm, various tracking steps, such as tracking steps in units of 100 frames or tracking steps in units of 1000 frames, are determined. As described above, the absolute value of the tracking step is based on the difference between the first frame number and the second frame number.

The image tracking unit 237, from the image frame of the first frame number to the image frame of the second frame number according to the tracking speed determined by the tracking algorithm selected by the algorithm selection unit 235, gate size, gate coordinates Track the movement of the target against. In the image tracking unit 237, when the image frame for the first frame number is searched by the frame search unit 233, and the tracking target is specified by the gate size and the gate coordinate, the tracking target is framed according to the passage of time. It is possible to determine whether the size and position of the change within the tracking speed according to the unique tracking speed defined in the tracking algorithm is included in the image captured from the EO-IR device. For example, assuming that t-100 is the first frame number, the image tracking unit 237 determines the tracking target according to the gate size and the gate coordinate in t-100, and then the algorithm selector 235 Depending on the selected tracking algorithm, the tracking target can be tracked to t-100, t-98, t-96, t-94, ... t (current time). In the above example, it can be seen that the tracking speed defined in the tracking algorithm is 2 frames/tracking. The tracking speed may be 1 frame/tracking number, 3 frames/tracking number, in addition to the above-described 2 frames/tracking number, and the tracking speed may be considered mathematically, empirically, and experimentally by comprehensively considering the efficiency of the tracking algorithm and the capture failure rate of the tracking target. It can be determined by.

In the present invention, first, in accordance with the image tracking information received from the ground control device 30, the image frame in which the tracking object corresponding to the gate size and the gate coordinates is taken is searched from the past image data, and the After the video frame is searched, the tracking target at the current time (current frame) is repeated by repeatedly tracking how the tracking target has been changed sequentially, taking into account the temporal term from the past time point to the current time point. It is possible to accurately track which gate size is located at which gate coordinate, and it is possible to expect an effect of not missing the tracking object primarily captured by the tracking command of the ground control device at a later time. In particular, the algorithm selection unit 235 selects an algorithm having an appropriate tracking speed through the difference between the first frame number and the second frame number, so that an appropriately fast speed is obtained without missing a tracking target from a past time point to a current time point. Can reach.

As an optional embodiment, the algorithm selection unit 235 may select any one of a first tracking algorithm having a first tracking speed and a second tracking algorithm having a second tracking speed, where The ratio of the first tracking speed may be a preset speed ratio.

In Equation 1, vt1 is the first tracking speed, vt2 is the second tracking speed, and Rtrack is the ratio of the first tracking speed to the second tracking speed. The algorithm selection unit 235 may set the ratio of the first tracking speed and the second tracking speed to a preset speed ratio. According to this exemplary embodiment, by setting the speed ratios of the first tracking speed and the second tracking speed to a specific ratio value in advance, it is possible to secure the efficiency or predictability of the tracking algorithm and fail to track the tracking target If you do, you can expect the effect of making it easier to adjust the tracking speed or to recover the cause of the tracking failure.

3 is a diagram schematically showing a process in which raw image data and metadata generated from EO-IR equipment are compressed by an image compression system.

For convenience of description, FIG. 3 will be described with reference to FIGS. 1 and 2.

The raw image data is separated for each frame in the IPA module 17. In this process, the metadata for the frame number is also separately managed. The IPA module 17 transfers the raw image data to the frame buffer of the CTA module 19 for each frame, and the number of frame buffers of the CTA module 19 is set equal to the number of frames of the raw image data. As shown in FIG. 3, the frame buffer may be implemented as an FPGA module (Field Programmable Gate Array Module) having a VDMA buffer equal to the number of frames. Metadata about the frame number is transferred to the SPI buffer of the FPGA module, not the VDMA buffer.

The FPGA module may include a GPIO signal transmitter 215 in FIG. 2. The GPIO signal is a multi-purpose input/output port signal, and in the present invention, functions as a signal for synchronizing a compressed image and a frame number. When the image data is transmitted from the FPGA module to the DSP module, the GPIO signal transmitter 215 generates a GPIO signal so that metadata (frame numbers, etc.) separately stored in the SPI buffer is transmitted to the ARM module. Specifically, the GPIO signal transmitter 215 transmits the SPI reading signal to the DSP module, so that meta information of the image data stored in the SPI buffer is accumulated in the data buffer queue of the ARM module. The metadata accumulated in the data buffer queue is synchronized from the compressed image and the ARM module and transferred to the SSD device. In this process, the video data divided and stored in frames in the VDMA buffer is compressed by a general-purpose encoder such as H.264, and then synchronized with the meta information of the data buffer queue. As described above, the CTA module 19 divides and processes data into an FPGA module, a DSP module, and an ARM module to maximize efficiency in processing each data, such as a slow external bus speed of the DSP module.

4 is a diagram schematically showing an example of an image tracking command transmission flow through the ground control device.

Referring to FIG. 4, a user may issue a video tracking command through a video playback device of the ground control device 30 by specifying a frame including a tracking target, and the video tracking command issued by the user is ground. The EO-IR device 10 is reached by the unmanned data link module 20 through the control device 30 so that the EO-IR device 10 performs image tracking for the tracking object through a series of processes. . As described above, the image tracking command includes information on the frame number, gate size, and location of gate, and the EO-IR device 10 regards the frame number as the aforementioned first frame number, Through the gate size and the gate coordinates, a tracking target corresponding to an image tracking command is determined from a frame of a past view stored in the image buffer, and tracking is performed according to a tracking speed set up to the current frame. That is, the image tracking system 230 starts tracking internally (image buffer) with the position (gate coordinate) and size (gate size) of the target received from the first frame number, and the frame number being tracked from the inside is being recorded. When the current frame number is reached, tracking results are started to be displayed in a normal tracking state. The process of tracking the image by the image tracking system 230 included in the EO-IR equipment 10 has been described in detail with reference to FIG. 2 and will be omitted.

FIG. 5 is a flowchart illustrating a process in which the image compression system compresses the raw image data and transmits it to the ground control device.

5 may be implemented by the image compression system 210 of FIG. 2, it will be described with reference to FIG. 2, and descriptions overlapping with those described in FIG. 2 will be omitted below.

First, the image data receiving unit 211 receives raw image data and a frame number (S510).

Subsequently, the image buffer storage unit 212 separates the received raw image data for each frame, and stores each in the frame buffer provided for each frame (S520).

The number buffer storage unit 213 stores the received frame number in the number buffer (S530).

The image compression unit 214 receives the raw image data stored in the frame buffer, and compresses it collectively (S540).

The synchronization processing unit 216 synchronizes the frame number stored in the number buffer for each frame of the compressed image (S550). Prior to step S550, the GPIO signal transmission unit 215 may transmit a multipurpose input/output port signal (GPIO signal) to the number buffer before transmitting the frames of the raw image data stored in the frame buffer to the image compression unit 214. It has already been described through FIG. 3.

FIG. 6 is a flowchart illustrating a process of tracking an image by an image tracking system that receives an image tracking command from a ground control device.

6 may be implemented by the image tracking system 230 of FIG. 2, it will be described with reference to FIG. 2, and descriptions overlapping with those described in FIG. 2 will be omitted below.

The tracking information receiving unit 231 receives image tracking information including a first frame number, gate size, and gate coordinates from the ground control equipment (S610).

The frame search unit 233 searches for a frame of a past view with a target in the image buffer based on the image tracking information (S620).

The algorithm selection unit 235 may select any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number (S630). As an optional embodiment of step S630, the algorithm selector 235 may select either a fast centroid tracking algorithm or a slow correlation tracking algorithm. Since the centroid tracking algorithm operates quickly, the size of the tracking speed expressed as a number has a small characteristic, and the correlation tracking algorithm is slow to track the target to be tracked with the current frame due to its slow-moving nature. It has characteristics. For example, the tracking speed of the centroid tracking algorithm is 1 frame/tracking, and the tracking speed of the correlation tracking algorithm is 2 frames/tracking.

The image tracking unit 237 tracks the movement of the target from the frame of the past viewpoint to the frame of the current viewpoint at a tracking speed determined by the tracking algorithm selected in step S630 (S640).

According to the present invention, it is possible to dramatically improve the initial success rate for the target by minimizing the difference between the image viewing time on the ground equipment and the shooting time of the EO-IR equipment of the unmanned aerial vehicle, and to a remotely controlled platform such as an unmanned aerial vehicle. By increasing the success rate of the installed EO-IR mission, the utilization of the EO-IR equipment can be further improved.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program can be recorded on a computer-readable medium. At this time, the medium includes a hard disk, a magnetic medium such as a floppy disk and magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floptical disk, and a ROM. , Hardware devices specially configured to store and execute program instructions, such as RAM, flash memory, and the like.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software field. Examples of computer programs may include not only machine language codes produced by a compiler, but also high-level language codes executable by a computer using an interpreter or the like.

The specific implementations described in the present invention are exemplary embodiments, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings are illustrative examples of functional connections and/or physical or circuit connections, and in the actual device, alternative or additional various functional connections, physical Connections, or circuit connections. In addition, unless specifically mentioned, such as “essential”, “importantly”, etc., it may not be a necessary component for application of the present invention.

In the specification (particularly in the claims) of the present invention, the use of the term “above” and similar indication terms may be in both singular and plural. In addition, in the case of describing a range in the present invention, as including the invention to which the individual values belonging to the range are applied (if there is no contrary description), each individual value constituting the range is described in the detailed description of the invention. Same as Finally, unless there is a clear or contradictory description of the steps constituting the method according to the invention, the steps can be done in a suitable order. The present invention is not necessarily limited to the description order of the above steps. The use of all examples or exemplary terms (eg, etc.) in the present invention is merely for describing the present invention in detail, and the scope of the present invention is limited due to the examples or exemplary terms, unless it is defined by the claims. It does not work. In addition, those skilled in the art can recognize that various modifications, combinations, and changes can be configured according to design conditions and factors within the scope of the appended claims or equivalents thereof.

Claims (9)

As an image processing method of an electro-optical-infrared (EO-IR) camera mounted on an unmanned aerial vehicle,
An image data receiving step of receiving the raw image data output from the camera and the frame numbers of the frames constituting the raw image data;
An image buffer storage step in which the image buffer storage unit separates the received raw image data for each frame, and stores each of the received raw image data in a frame buffer provided for each frame;
A number buffer storage step in which the number buffer storage unit stores the received frame number in a number buffer;
An image compression step in which the image compression unit receives the raw image data stored in the frame buffer and collectively compresses it; And
Synchronization processing step of synchronizing the frame number stored in the number buffer by the synchronization processing unit for each frame of the compressed image; A method for processing an image of an electro-optical-infrared (EO-IR) camera mounted in an unmanned aerial vehicle. According to claim 1,
The treatment method,
The GPIO signal transmission step further comprises transmitting a multipurpose input/output port (GPIO) signal to the number buffer before transmitting the frames of the raw image data stored in the frame buffer to the image compression unit. Electro-optical-infrared (EO-IR) camera image processing method. As an image tracking method of an electro-optical-infrared camera mounted on an unmanned aerial vehicle,
A tracking information receiving step of receiving image tracking information including a first frame number, gate size, and gate coordinates from the ground equipment;
A frame search step of searching for a video frame for the first frame number in an image buffer if the received first frame number is a number of a frame at a point earlier than the second frame number for the current time;
An algorithm selection step of selecting any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number; And
An image tracking step of tracking the movement of the target with respect to the gate size and the gate coordinates from the video frame of the first frame number to the video frame of the second frame number according to the tracking speed determined by the selected tracking algorithm. Image tracking method of an electro-optical-infrared (EO-IR) camera mounted on an unmanned aerial vehicle comprising a. According to claim 3,
The algorithm selection step,
Selecting one of a first tracking algorithm having a first tracking speed and a second tracking algorithm having a second tracking speed,
The ratio of the first tracking speed to the second tracking speed is a predetermined speed ratio, characterized in that the electronic optical-infrared (EO-IR) camera image tracking method mounted on an unmanned aerial vehicle. A computer-readable recording medium storing a program for executing the method according to any one of claims 1 to 4. An image processing system of an EO-IR camera mounted on an unmanned aerial vehicle,
An image data receiving unit that receives the raw image data output from the camera and frame numbers of frames constituting the raw image data;
An image buffer storage unit for separating the received raw image data for each frame and storing each of the received raw image data in a frame buffer provided for each frame;
A number buffer storage unit for storing the received frame number in the number buffer;
An image compression unit for receiving the raw image data stored in the frame buffer and compressing it in a batch; And
Synchronization processing unit for synchronizing the frame number stored in the number buffer frame by frame of the compressed image; an image processing system of an electro-optical-infrared (EO-IR) camera mounted in an unmanned aerial vehicle comprising a. The method of claim 6,
The processing system,
An electronic device mounted on an unmanned aerial vehicle further comprising a GPIO signal transmitting unit, characterized in that a frame of the raw image data stored in the frame buffer is transmitted to the number buffer by sending a multipurpose input/output port (GPIO) signal to the image compression unit. Optical-infrared (EO-IR) camera image processing system. As an image processing system of an electro-optical-infrared camera mounted on an unmanned aerial vehicle,
A tracking information receiver that receives image tracking information including a first frame number, gate size, and gate coordinates from ground equipment;
A frame search unit for searching the video frame for the first frame number in an image buffer if the received first frame number is a frame number at a point earlier than the second frame number for the current time;
An algorithm selection unit selecting any one of a plurality of tracking algorithms based on the difference between the first frame number and the second frame number; And
Depending on the tracking speed determined by the selected tracking algorithm, the image tracking unit tracks the movement of the target with respect to the gate size and the gate coordinates from the video frame of the first frame number to the video frame of the second frame number. An image tracking system of an electro-optical-infrared (EO-IR) camera mounted on an unmanned aerial vehicle. The method of claim 8,
The algorithm selection unit,
Selecting one of a first tracking algorithm having a first tracking speed and a second tracking algorithm having a second tracking speed,
The ratio of the first tracking speed to the second tracking speed is an image processing system of an electro-optical-infrared (EO-IR) camera mounted in an unmanned aerial vehicle, characterized in that a predetermined speed ratio.

Images