KR102279730B1 - MULTI-TRANSMISSION STREAMING METHOD OF IMAGES WITH META INFORMATION BASED ON REAL-TIME IoT, RECORDING MEDIUM, DEVICE AND SYSTEM FOR PERFORMING THE METHOD - Google Patents

Abstract

A multi-transmission streaming method for an image includes: encoding image data received from an image sensor; a packetizing step of generating an image data packet in one transmission unit including multi-divisions, a picture number, a packet number, and data to transmit an encoded byte stream; configuring a network module to perform communication by simultaneously using LTE and a wireless network including a first Wi-Fi and a second Wi-Fi, which have mutually different bandwidths, to perform multi-transmission of the image data packet; a packet scheduling step of analyzing a signal-to-noise ratio of each of the wireless networks to add at least one wireless network among the LTE, the first Wi-Fi, and the second Wi-Fi to a packet scheduler as a wireless network; determining, when a plurality of wireless networks are added to the packet scheduler, an image data packet to be transmitted for each of the added wireless networks; and transmitting, by the wireless network, only the image data packet to the wireless network added to the packet scheduler. Accordingly, an image is stably received from a mobile camera.

Description

Multi-transmission streaming method of video having real-time IoT-based meta information, recording medium, apparatus and system for performing the same THE METHOD}

The present invention relates to a multi-transmission streaming method of an image having real-time IoT-based meta information, a recording medium, an apparatus and a system for performing the same, and more particularly, disasters/disasters such as steep slopes and rivers with poor network environments In order to provide a real-time transmission service of an image having meta information of an IP-based camera that is used in an area or mounted on a moving object such as an unmanned aerial vehicle, a ship, or a vehicle (firefighting, police, etc.) It is for streaming service transmitted over multi-channel wireless Internet network.

Recently, convergence of each field is taking place not only in the domestic market but also in the overseas market, and with the advent of smart phones, text-based services are rapidly changing to video-based. In this environment, while the existing CCTV video surveillance is mainly limited to central monitoring of the inside of a building or remote monitoring with a PC, the demand for real-time monitoring of camera images through moving objects is gradually increasing.

In addition, with the rapid development of wireless mobile communication network infrastructure and the development of smart phones and tablet PCs, the demand for simultaneous real-time access is increasing, and Internet-of-things-type micro-image stabilization processing is required through the development of high-performance embedded CPUs.

For example, there is an increasing demand for automatically generating stereoscopic images based on images acquired from video cameras and smart phones or regenerating immersive virtual reality (VR) images viewed from a different point of view in a new path. are doing

In particular, recently, there is an increasing demand for using drones to produce images that view objects from a point of view that has not been experienced before. Accordingly, studies related to aerial photography using drones are being actively conducted. In addition, images are being taken in various fields for vehicle, ship, military, aviation, harbor, forest fire monitoring, building monitoring, and the like.

Accordingly, it should be possible to perform real-time control and reconnaissance by mounting video equipment on a moving object and transmitting video information without persistent video in real time.

KR 10-2016-0072971 A KR 10-2016-0070922 A

Accordingly, the technical problem of the present invention has been conceived in this regard, and an object of the present invention is to provide a multi-transmission streaming method of an image having real-time IoT-based meta information.

Another object of the present invention is to provide a recording medium in which a computer program for performing the multi-transmission streaming method of an image having the real-time IoT-based meta information is recorded.

Another object of the present invention is to provide an apparatus for performing a multi-transmission streaming method of an image having the real-time IoT-based meta information.

Still another object of the present invention is to provide a system including a multi-transmission streaming apparatus for video for a multi-transmission streaming service of video having the real-time IoT-based meta information.

According to an embodiment of the present invention, there is provided a method for multiple transmission and streaming of an image, comprising: encoding image data received from an image sensor; a packetizing step of generating an image data packet in one transmission unit including multiple divisions, a picture number, a packet number, and data to transmit the encoded byte stream; configuring a network module to communicate simultaneously using a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet; a packet scheduling step of analyzing the signal-to-noise ratio of each wireless network and adding at least one wireless network among the LTE, the first WiFi, and the second WiFi as a wireless network to a packet scheduler; determining an image data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and transmitting, by the wireless network, only video data packets, including transmitting the video data packets to a wireless network added to the packet scheduler.

In an embodiment of the present invention, the packet scheduling step of adding the at least one wireless network as a wireless network to a packet scheduler includes: analyzing a signal-to-noise ratio of each wireless network; adding only the first Wi-Fi to the packet scheduler as a wireless network when the signal-to-noise ratio of the first Wi-Fi is equal to or greater than a first threshold; adding only the second Wi-Fi to the packet scheduler as a wireless network when the signal-to-noise ratio of the second Wi-Fi is equal to or greater than the first threshold; adding the first Wi-Fi and the LTE as a wireless network to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a second threshold and less than a first threshold; and if the signal-to-noise ratio of the second Wi-Fi is equal to or greater than the second threshold, a packet scheduling step of adding the second Wi-Fi and the LTE to the packet scheduler as a wireless network.

In an embodiment of the present invention, the step of determining the video data packet to be transmitted for each added wireless network includes: among all video data packets, when the added wireless network is one of the first Wi-Fi and the second Wi-Fi and is the LTE. An image data packet including a packet number selected according to a preset condition may be determined as an image data packet to be transmitted over the LTE, and the remaining image data packets may be determined as an image data packet to be transmitted through the first Wi-Fi or the second Wi-Fi.

In an embodiment of the present invention, the step of determining the video data packet to be transmitted for each added wireless network includes: when the added wireless network is the first Wi-Fi, the second Wi-Fi, and the LTE, preset image data packets among all the video data packets An image data packet including a picture number selected according to a condition is determined as an image data packet to be transmitted to the LTE, and an image data packet including a preset packet number among the remaining image data packets is selected from among the first Wi-Fi and the second Wi-Fi. It may be determined as an image data packet to be transmitted through Wi-Fi having a higher priority, and the remaining image data packets may be determined as image data packets to be transmitted through Wi-Fi having a lower priority.

In an embodiment of the present invention, the priority may be set in the order of transmission speed, saturation, and signal interference.

In an embodiment of the present invention, the multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, and the picture number represents one frame of an image encoded at 30 frames per second. It is indicated as a picture of , and includes number information for distinguishing each picture, and the packet number may include packet number information for distinguishing each packet divided into n packets according to the size of one picture.

In an embodiment of the present invention, the first Wi-Fi may support a bandwidth of 24 GHz, and the second Wi-Fi may support a bandwidth of 50 GHz.

In an embodiment of the present invention, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

In an embodiment of the present invention, encoding the image data received from the image sensor may encode the image data input from the sensor into an H264 codec format.

In a computer-readable storage medium on which computer programming is recorded according to an embodiment for realizing another object of the present invention, a computer program for performing a method of multiple transmission and streaming of an image is recorded.

According to an embodiment of the present invention for realizing another object of the present invention, there is provided a multi-transmission streaming apparatus for an image provided in an IP-based mobile device, comprising: an encoding unit for encoding image data received from an image sensor; a packetizing unit for generating an image data packet in one transmission unit including multiple divisions, picture numbers, packet numbers, and data to transmit the encoded byte stream; a multiple wireless modem supporting a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet; By analyzing the signal-to-noise ratio of each wireless network, if the signal-to-noise ratio of the first WiFi is equal to or greater than a first threshold value, only the first WiFi is added as a wireless network to the packet scheduler, and the signal-to-noise ratio of the second WiFi is the first If it is equal to or greater than a threshold, only the second Wi-Fi is added to the packet scheduler as a wireless network, and the first Wi-Fi and the second Wi-Fi having a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi equal to or greater than a second threshold and less than a first threshold. a packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network; a packet determiner configured to determine a video data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and a packet multiplexing unit transmitting the video data packet to the wireless network added to the packet scheduler.

In an embodiment of the present invention, when the wireless network added in the packet scheduler is one of the first Wi-Fi and the second Wi-Fi and the LTE is the LTE, the packet determiner is configured to select a packet selected according to a preset condition from among all image data packets. An image data packet including a number may be determined as an image data packet to be transmitted through the LTE, and the remaining image data packets may be determined as an image data packet to be transmitted through the first Wi-Fi or the second Wi-Fi.

In an embodiment of the present invention, when the wireless networks added in the packet scheduler are the first Wi-Fi, the second Wi-Fi, and the LTE, the packet determiner is configured to select a picture number selected by a preset condition from among all image data packets. Determines an image data packet including an image data packet to be transmitted to the LTE, and transmits an image data packet including a preset packet number among the remaining image data packets to a higher-priority Wi-Fi among the first Wi-Fi and the second Wi-Fi. It is determined as an image data packet, and the remaining image data packets may be determined as image data packets to be transmitted over the low-priority WiFi.

In an embodiment of the present invention, the priority may be set to be high in the order of transmission speed, saturation, and signal interference.

In an embodiment of the present invention, the multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, and the picture number represents one frame of an image encoded at 30 frames per second. It is indicated as a picture of , and includes number information for distinguishing each picture, and the packet number may include packet number information for distinguishing each packet divided into n packets according to the size of one picture.

In an embodiment of the present invention, the first Wi-Fi may support a bandwidth of 24 GHz, and the second Wi-Fi may support a bandwidth of 50 GHz.

In an embodiment of the present invention, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

In an embodiment of the present invention, the encoding unit may encode the image data input from the sensor in the H264 codec format.

A system according to an embodiment for realizing another object of the present invention is a streaming server for receiving video data packets transmitted from a multi-transmission streaming device of an image provided in an IP-based mobile device and the multi-transmission streaming device. Including, the multi-transmission streaming apparatus of the image, the encoding unit for encoding the image data received from the image sensor; a packetizing unit for generating an image data packet in one transmission unit including multiple divisions, picture numbers, packet numbers, and data to transmit the encoded byte stream; a multiple wireless modem supporting a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet; By analyzing the signal-to-noise ratio of each wireless network, if the signal-to-noise ratio of the first WiFi is equal to or greater than a first threshold value, only the first WiFi is added as a wireless network to the packet scheduler, and the signal-to-noise ratio of the second WiFi is the first If it is equal to or greater than a threshold, only the second Wi-Fi is added to the packet scheduler as a wireless network, and the first Wi-Fi and the second Wi-Fi having a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi equal to or greater than a second threshold and less than a first threshold. a packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network; a packet determiner configured to determine a video data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and a packet multiplexing unit transmitting the video data packet to the wireless network added to the packet scheduler.

In an embodiment of the present invention, the streaming server includes: a multi-packet receiver for storing the video data packets transmitted by the multi-transmission streaming in a database according to the arrival time; Packet reconstruction module that selects the image data stored in the database first according to the arrival priority time and discards the unselected image data when the multiple classification values are different for image data having the same picture number and packet number in the stored image multiplex data ; and a packet transmitter configured to transmit image data by removing multi-classification and packet number fields when transmitting the reconstructed packet to the user terminal.

According to this multi-transmission streaming method of video with real-time IoT-based meta information, it provides stable wired/wireless communication connection by collaborating with operators of LTE/LTE-m and WiFi network networks, which are next-generation wireless communications in commercial service, Through a cloud-based streaming service, stable video can be provided to customers in real time. In particular, it is possible to receive stable images even in low data band fluctuations in LTE (private wireless network).

1 is a conceptual diagram of a multi-transmission streaming system for multi-transmission streaming of an image according to an embodiment of the present invention.
2 is a block diagram of a multiple transport streaming system for multiple transport streaming of an image according to an embodiment of the present invention.
3 is a conceptual diagram of a TCP/IP segment and a transport packet for multi-transmission of an image according to the present invention.
4 is a conceptual diagram of packet reconstruction according to packet loss and arrival priority time.
5 is a flowchart of a multi-transmission streaming method for multi-transmission streaming of an image according to an embodiment of the present invention.
6 is a detailed flowchart of packet scheduling of FIG. 5 .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0023] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the invention may be practiced. These examples are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. In addition, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the invention, if properly described, is limited only by the appended claims, along with all scope equivalents to those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

First, FIG. 1 is a conceptual diagram of a multiple transport streaming system for multiple transport streaming of an image according to an embodiment of the present invention, and FIG. 2 is a multiple transport streaming for multiple transport streaming of an image according to an embodiment of the present invention. It is a block diagram of the system.

The multi-transmission streaming system (1, hereinafter system) of the video according to the present invention includes a video multi-transmission streaming device (10, hereinafter device), a streaming server 30, and a user terminal 50, the device 10 The image data received from the streaming server 30 is stably provided to the user terminal 50 .

The above image data refers to data collected from the device 10. In an embodiment of the present invention, the image data has been described as a reference, but data collected from a collection sensor based on the Internet of Things, including sound. have.

The device 10 may be provided or communicated with an IP-based mobile camera that collects image data, as well as provided with or communicated with an IP-based fixed camera based on the Internet of Things (packet number). . For example, the device 10 may be included in imaging equipment mounted on a drone, vehicle, ship, military, aviation, port, or the like.

In the present invention, in order to stably provide a real-time transmission service of an image of an IP-based camera mounted on a moving object, the captured image is transmitted multiple times through LTE and Wi-Fi.

1 and 2 , the apparatus 10 according to the present invention includes an encoding unit 110 , a packetizing unit 120 , a multi-radio modem 130 , a packet scheduler 140 , and a packet determining unit 150 . and a packet multiplex transmission unit 160 .

In the device 10 of the present invention, software (application) for performing multiple transmission streaming of an image having real-time IoT-based meta information may be installed and executed, and the encoding unit 110 and the packetizing unit 120 may be installed. , the configuration of the multi-radio modem 130 , the packet scheduler 140 , the packet determiner 150 , and the packet multi-transmission unit 160 is the real-time IoT-based meta information executed in the device 10 . It can be controlled by software for performing multi-transmission streaming of an image with

The device 10 may be a separate terminal or a module of the terminal. In addition, the encoding unit 110 , the packetizing unit 120 , the multiple wireless modem 130 , the packet scheduler 140 , the packet determining unit 150 , and the packet multiplexing unit 160 are configured may be formed as an integrated module, or may consist of one or more modules. However, on the contrary, each configuration may be formed of a separate module.

The device 10 may be movable or stationary. The apparatus 10 may be in the form of a server or an engine, and may be a device, an application, a terminal, a user equipment (UE), a mobile station (MS), or a wireless device. (wireless device), may be called other terms such as a handheld device (handheld device).

The device 10 may execute or manufacture various software based on an operating system (OS), that is, the system. The operating system is a system program for software to use the hardware of the device, and is a mobile computer operating system such as Android OS, iOS, Windows Mobile OS, Bada OS, Symbian OS, Blackberry OS, and Windows series, Linux series, Unix series, It can include all computer operating systems such as MAC, AIX, and HP-UX.

The encoding unit 110 encodes image data received from an image sensor (eg, a camera). The encoding unit 110 may encode the input image data in a codec format, for example, in an H.264 format.

The packetizing unit 120 generates an image data packet in one transmission unit to transmit the encoded byte stream.

3 is a conceptual diagram of a TCP/IP segment and a transport packet for multi-transmission of image data according to the present invention. As shown in FIG. 3, the packetizing unit 120 includes a 40-byte TCP/IP header and a payload. By adding 1460 bytes, 1500 bytes can be set as one transmission unit. 1460 bytes of the payload include 1-byte multiple division, 2-byte picture number, 2-byte packet number, and 1455 bytes of data.

The multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, and the picture number displays one frame of an image encoded at 30 frames per second as one picture, and each picture and number information for distinguishing one picture, and the packet number includes packet number information for distinguishing each packet in which one picture is divided into n packets according to the size.

In this case, the packet number may be assigned based on a preset standard.

For example, if one picture is a rectangle and one picture is configured in a 9x9 grid format according to the size, all packets located in the first row start with 1 and consecutive numbers are assigned in the order of the column. can

Specifically, the packet number of the packet located in the first column of the first row is 101, the packet number of the packet located in the second column of the first row is 102, and the packet number of the packet located in the multi-line column of the fifth row is 505.

Of course, this is only a specific example and is not limited thereto, and a packet number may be assigned to each packet by providing appropriate packet number setting criteria if necessary, or a packet number may be sequentially assigned to each packet.

The multi-wireless modem 130 is a network module for communication using a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for double transmission of the generated image data packet at the same time. make up

The multiple wireless modem 130 configures a network module so that the device 10 can communicate using three wireless networks at the same time.

The first Wi-Fi may support a bandwidth of 24 GHz, and the second Wi-Fi may be a wireless network supporting a bandwidth of 50 GHz. In case of Wifi 24GHz, the range (coverage) is wide, but since it is almost saturated, there may be signal interference. Wifi5GHz is faster than Wifi 24GHz, but its range (coverage) is narrow and its straightness is strong, so it is weak at corners.

However, this is only an example, and if necessary, a wireless network supporting another bandwidth or a wireless network supporting another bandwidth may be added.

The packet scheduler 140 analyzes the sensitivity of each wireless network to maximize the transmission success rate and reduce the resources of the network camera and the streaming server, and schedules the optimal packet according to the signal-to-noise ratio sensitivity.

To this end, the packet scheduler 140 analyzes the signal-to-noise ratio of each wireless network, and adds at least one wireless network among the LTE, the first WiFi, and the second WiFi as a wireless network to the packet scheduler.

Specifically, if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to the first threshold, only the first Wi-Fi is added as a wireless network to the packet scheduler, and if the signal-to-noise ratio of the second Wi-Fi is greater than or equal to the first threshold, the second Wi-Fi Only add it as a wireless network to the packet scheduler.

On the other hand, at least one of the first Wi-Fi and the second Wi-Fi and the LTE with a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi that is less than the first threshold and greater than or equal to the second threshold are transmitted to the packet scheduler as a wireless network. add

In other words, if the signal-to-noise ratio of the first Wi-Fi is less than the first threshold and greater than or equal to the second threshold, the first Wi-Fi and the LTE are added to the packet scheduler as wireless networks.

In addition, if the signal-to-noise ratio of the second Wi-Fi is less than the first threshold and greater than or equal to the second threshold, the second Wi-Fi and the LTE are added to the packet scheduler as a wireless network.

If the signal-to-noise ratios of the first Wi-Fi and the second Wi-Fi are both less than the first threshold and greater than or equal to the second threshold, the first Wi-Fi and the second Wi-Fi and the LTE are both sent to the packet scheduler as a wireless network is added to

For example, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

Meanwhile, when a plurality of wireless networks are added to the packet scheduler 140 , the packet determiner 150 may determine an image data packet to be transmitted for each added wireless network.

To this end, when the added wireless network is one of the first Wi-Fi and the second Wi-Fi and is the LTE, an image data packet including a packet number selected according to a preset condition among all generated image data packets is transmitted to the LTE. It is determined as the image data packet to be transmitted, and the remaining image data packets are determined as the image data packet to be transmitted through the first Wi-Fi or the second Wi-Fi.

Here, the preset condition means that when a frame is divided into n packets, the packet to be transmitted is determined by assigning importance according to the location of the packet based on the purpose of collecting the image data, the type of image data, etc. The image data packet to be transmitted is determined based on the total number of packets included in one picture.

This is to reduce the time required to reconstruct the packet by transmitting the image data packet without overlap according to the setting, unlike the embodiment in which the packet reconstruction module 330 described later reconstructs the stored packet according to whether or not there is overlap.

For example, it may be implemented when it is desired to receive low-resolution image data without a delay time when receiving image data according to the network state and settings of the user terminal 50 .

Specifically, when a picture is a rectangle and a picture is configured in a 9x9 grid format according to the size of a picture as in the example described above, the packets located in the first row all start with 1 and continue in the order of the columns. In this way, the packet number of the packet located in the first column of the first row is 101, the packet number of the packet located in the second column of the first row is 102, and the packet number of the packet located in the multi-line column of the fifth row is 505. A case in which the method is performed will be described as an example.

If the added wireless network is the first Wi-Fi and the LTE, and the importance of the image data packet including the packet number located in the 5th row to the 8th row in one picture of the image data is high. In this case, video data packets including packet numbers 501 to 809 among all generated video data packets are determined as video data packets to be transmitted to LTE, and packet numbers 101 to 409 and 901 to 909, which are the remaining packet numbers, are determined. The included image data packet may be determined as an image data packet to be transmitted through the first Wi-Fi.

Alternatively, if the added wireless network is the first Wi-Fi and the LTE, and the packet is simply divided and transmitted regardless of the importance of the packet, the even columns 102, 104, ..., 906, 908 of the odd rows and the even rows Video data packets including packet numbers in odd columns (201, 203, ..., 807,809) are determined as video data packets to be transmitted to LTE, and the remaining packet numbers are odd columns (101, 103, ..., 907, 909) in odd rows. ) and even-numbered columns 202, 204, ..., 806, 808 of even-numbered rows, the image data packet including the packet numbers may be determined as the image data packet to be transmitted through the first Wi-Fi.

Also, when the added wireless network is the first Wi-Fi, the second Wi-Fi, and the LTE, the packet determiner 150 is configured to include an image data packet including a picture number selected according to a preset condition among all image data packets. may be determined as the image data packet to be transmitted to the LTE.

In addition, the packet determiner 150 selects an image data packet including a packet number selected according to a preset condition from among the image data packets not including the selected picture number among the first Wi-Fi and the second Wi-Fi having a higher priority. It can be determined by the image data packet to be transmitted over Wi-Fi.

Also, the packet determiner 150 may determine the remaining image data packets to be image data packets to be transmitted over Wi-Fi having a lower priority.

Here, the preset condition for selecting the picture number may be determined according to the order of the pictures based on the purpose of collecting the image data, the type of image data, etc. when one frame of the image is displayed as one picture.

Specifically, the preset picture number selection condition is a picture number corresponding to an odd number, and for the remaining picture numbers, packets are simply divided and transmitted as described above. For example, picture numbers corresponding to odd numbers (1, 3, All video data packets including ..., 27, 29) are determined as video data packets to be transmitted to LTE, and video data packets including even picture numbers (2, 4, ..., 28, 30) are again packets. The video data packet including the packet numbers corresponding to the even columns (102,104, ..., 906, 908) of odd rows and odd columns (201, 203, ..., 807,809) of even rows based on the number is image data to be transmitted over the first Wi-Fi. Video data including packet numbers that are determined as packets and are odd columns (101, 103, ..., 907, 909) in odd rows and even columns (202, 204, ..., 806, 808) in even rows, which are the remaining packet numbers The packet may be determined as an image data packet to be transmitted through the second Wi-Fi.

The above example is a case in which transmission is performed irrespective of the priorities of the first Wi-Fi and the second Wi-Fi, and the priorities will be described later.

Alternatively, depending on the purpose of the image to be collected, if the importance of a frame indicated by a picture number corresponding to a multiple of 5 among images composed of 30 frames per second is high, the image data packet including a picture number corresponding to a multiple of 5 is It is also possible to determine the image data packet to be transmitted through LTE, and determine the image data packet including the remaining picture number as the image data packet to be transmitted through the first Wi-Fi and the second Wi-Fi based on the packet number again.

In addition, when the video data packets other than the video data packet transmitted to LTE are transmitted over the first Wi-Fi and the second Wi-Fi, as described above, if the importance of the packet located at a specific position in the frame is set to be high, the importance The video data packet including the high packet number is transmitted through high-priority Wi-Fi.

In the present invention, the priority of the first Wi-Fi and the second Wi-Fi may be set to be high in the order of transmission speed, saturation, and signal interference.

Therefore, the packet determiner 150 prioritizes the transmission speed and, when the transmission speed of the first WiFi is faster than that of the second WiFi, an image to transmit an image data packet including a packet number with high importance to the first WiFi. An image data packet determined as a data packet and not including a packet number having high importance may be determined as an image data packet to be transmitted through the second Wi-Fi.

In addition, the packet determiner 150 compares the saturation degree, which is the next priority, when the difference in the transmission speed is within a predetermined range as a result of comparing the transmission rates of the first WiFi and the second WiFi.

As a result of the comparison of the saturation, if the saturation of the second Wi-Fi is lower than that of the first Wi-Fi, the packet determiner 150 determines an image data packet including a packet number having a high importance as an image data packet to be transmitted to the second Wi-Fi, and the importance is An image data packet that does not include a high packet number may be determined as an image data packet to be transmitted through the first Wi-Fi.

In addition, when the difference in saturation is within a predetermined range as a result of comparing the saturation levels between the first WiFi and the second WiFi, the packet determiner 150 compares the signal interference level, which is the next priority.

When the signal interference of the second Wi-Fi is lower than that of the first Wi-Fi, the packet determination unit 150 determines the image data including the packet number having a high importance as the image data packet to be transmitted to the second Wi-Fi, and the packet number having the high importance. An image data packet that does not include may be determined as an image data packet to be transmitted through the first Wi-Fi.

If the comparison results of the transmission speed, saturation, and signal interference between the first WiFi and the second WiFi are all within a predetermined range, the packet determination unit 150 sets the highest priority based on the highest transmission speed of the WiFi wave having a high transmission rate. Accordingly, it can be determined to transmit the image data packet including the packet number having high importance.

In another embodiment, the packet determiner 150 compares the comparison results between priorities when the comparison results of the transmission speed, saturation, and signal interference between the first WiFi and the second WiFi are all within a predetermined range, and the comparison value It is possible to determine to transmit an image data packet including a packet number with high importance to the corresponding Wi-Fi by selecting the priority corresponding to the item with the greatest difference by comparing them.

Specifically, a criterion for judging that there is no difference in the comparison result between the transmission speed, saturation and signal interference between the first WiFi and the second WiFi is 10%, and the difference in transmission speed between the first WiFi and the second WiFi is 5%, the saturation is 8%, and the signal interference degree is 4%, the packet determination unit 150 is the highest value, and the packet number with high importance is the Wi-Fi having a lower saturation based on the difference in saturation. It may be determined to transmit an image data packet including

In another embodiment, when the comparison results of the transmission speed, saturation, and signal interference between the first Wi-Fi and the second Wi-Fi are all within a predetermined range, the packet determiner 150 selects image data including a packet number with high importance. One of the first Wi-Fi and the second Wi-Fi is randomly selected and determined as an image data packet to be transmitted over the selected Wi-Fi, and image data that does not include a packet number with high importance may be determined as an image data packet to be transmitted over the unselected Wi-Fi. .

Of course, the weight of these priorities may be changed based on the environment in which the device 10 is installed. In addition, a configuration for detecting, comparing, and determining the transmission rate, saturation, and signal interference may be provided in the packet determiner 150 , may be provided in the packet scheduler 170 , or may be configured as a separate module.

Accordingly, in the present embodiment, since it is possible to transmit the image data packets without overlapping packet numbers, the packet reconstruction module 330 simply collects all stored image data packets, thereby reducing the time required for reconstructing the image data packets. can be shortened.

In addition, since each wireless network transmits only the selected video data packet for each wireless network, not the entire video data packet, the amount of data to be transmitted for each wireless network is also reduced, thereby reducing the time required for data transmission. will be able to do

In addition, the packet determiner 150 may determine the entire generated image data packet as the image data packet to be transmitted for each added wireless network.

On the other hand, in the packet determiner 150 according to another embodiment of the present invention, when the added wireless network is the LTE, the first Wi-Fi, and the second Wi-Fi, and the importance of a frame indicated by a picture number is set differently, An entire data packet may be determined as an image data packet to be transmitted to the LTE, and only an image data packet including a picture number having high importance may be determined as an image data packet to be transmitted as the image data packet to be transmitted through the first Wi-Fi and the second Wi-Fi.

For example, if the picture number corresponding to a multiple of 5 has high importance, the entire data packet is determined as the video data packet to be transmitted to the LTE, and picture numbers corresponding to multiples of 5 (5, 10, 15, 20, 25, 30) are determined as data packets to be transmitted to the first Wi-Fi and the second Wi-Fi.

Meanwhile, the packet multiplexing unit 160 transmits the image data packet to the wireless network added to the packet scheduler.

The streaming server 30 is a cloud platform 300 that provides a cloud-based simultaneous multi-transmission platform (LTE, WiFi), a multi-packet receiver 310, a packet reconstruction module 330, and a packet transmitter 350 includes In addition, the streaming server 30 may further include a mobile/terminal connection state management 360 , log management and simultaneous connection management 370 , and an OS (operating system) 380 such as Linux or Windows.

The multi-packet receiving unit 310 stores the video data packets transmitted by the multi-transmission streaming in the database 320 according to arrival times.

The packet reconstruction module 330 first selects the image data stored in the database according to the arrival priority time when the multiple classification values are different for image data having the same picture number and packet number in the stored image multiplex data, and then selects the unselected image data. The video data is discarded.

Meanwhile, FIG. 4 is a conceptual diagram of packet reconstruction according to packet loss and arrival priority time.

As shown in FIG. 4 , the packet reconstruction module 330 indicates that the same picture number and the packet number in the image data stored in the database 320 indicate the same image data in the image stream, so the The video data with the least reception delay is selected from the video data, and the unselected video data is discarded.

Accordingly, in the reconstructed packet shown in FIG. 4, a picture number and a packet number constituting each picture number are arranged in the order, and referring to information on multiple classifications for classifying wireless networks, packets received in LTE and Wi-Fi are mixed It can be seen that it is composed of

Accordingly, the selected image data is collected, reconstructed into a new image stream, and transmitted to the user terminal 50 .

On the other hand, unlike in FIG. 4 , the packet reconstruction module 330 selects all the image multiplex data stored in the database 320 when the multiplex classification value, the picture number, and the packet number are all different in the stored image multiplex data. can do.

This may be a case in which the image data packets transmitted in the process of determining the image data packets transmitted for each network added by the packet determiner 150 are transmitted without overlapping.

In addition, in the packet reconstruction module 330 according to another embodiment of the present invention, the image data packet including the picture number having high importance in the image data stored in the database 32 is repeatedly received. The video data with the least reception delay is selected from the video data, the unselected video data is discarded, and the video data packets that do not include a picture number with high importance are not repeatedly received, so they can be collected and reconstructed without a separate discard process. have.

When transmitting the reconstructed packet to the user terminal 50 , the packet transmission unit 350 removes the multiple division and packet number fields to transmit image data.

In order to reduce network bandwidth, the packet transmitter 350 removes the multiplex division and packet number used for multiplexing during packetizing in the device 10 so that more image data can be transmitted.

On the other hand, if the image data packet transmitted by the wireless network added by the packet determiner 150 is not received in duplicate, the image multiplex data stored in the database 320 is transferred to the packet reconstruction module 330 as necessary. The image data may be transmitted to the user terminal 50 by removing only the multi-classification and packet number fields through the packet transmission unit 350 without going through a process of being reconstructed by .

The user terminal 50 may include a wireless modem 510 supporting a wireless network, a packet receiving unit 530 and a decoding unit 550 to stably receive the video received from the streaming server 30 .

Accordingly, it is possible to support stable video use to customers in real time through a cloud-based streaming service. In particular, it is possible to receive stable images even in low data band fluctuations in LTE (private wireless network).

FIG. 5 is a flowchart of a method for multi-transmission streaming of an image according to an embodiment of the present invention, and FIG. 6 is a detailed flowchart of packet scheduling of FIG. 5 .

The multi-transmission streaming method for the video stabilization service according to the present embodiment may proceed in substantially the same configuration as the apparatus 10 of FIG. 1 . Accordingly, the same components as those of the device 10 of FIG. 1 are given the same reference numerals, and repeated descriptions are omitted. In addition, the multi-transmission streaming method for the video stabilization service according to the present embodiment may be executed by software (application) for performing multi-transmission streaming for the video stabilization service.

In the present invention, in order to stably provide a real-time transmission service of an image of an IP-based camera mounted on a moving object, the captured image is transmitted twice through LTE and Wi-Fi. The moving object may be a drone, a vehicle, a ship, a military, an aviation, a port, and the like.

5 , in the multi-transmission streaming method for an image stabilization service according to the present embodiment, image data received from an image sensor is encoded (step S10). In this case, the input image data can be encoded in a codec format. and, for example, may be encoded in H.264 format.

In order to transmit the encoded byte stream, packetizing is performed to generate an image data packet in one transmission unit including multiple division, picture number, packet number, and data (step S20)

By adding 40 bytes of TCP/IP header and 1460 bytes of payload, 1500 bytes can be set as one transmission unit. 1460 bytes of the payload include 1-byte multiple division, 2-byte picture number, 2-byte packet number, and 1455 bytes of data.

The multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, and the picture number displays one frame of an image encoded at 30 frames per second as one picture, and each picture number information for distinguishing one picture, and the packet number includes packet number information for distinguishing each packet in which one picture is divided into n packets according to the size (refer to FIG. 3).

For double transmission of the generated image data packet, a network module is configured to communicate using a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE at the same time. (Step S30)

At this time, the network module is configured to communicate using three wireless networks simultaneously.

The first Wi-Fi may support a bandwidth of 24 GHz, and the second Wi-Fi may be a wireless network supporting a bandwidth of 50 GHz. In case of Wifi 24GHz, the range (coverage) is wide, but since it is almost saturated, there may be signal interference. Wifi5GHz is faster than Wifi 24GHz, but its range (coverage) is narrow and its straightness is strong, so it is weak at corners.

However, this is only an example, and if necessary, a wireless network supporting another bandwidth or a wireless network supporting another bandwidth may be added.

By analyzing the signal-to-noise ratio of each wireless network, packet scheduling is performed by adding at least one wireless network among the LTE, the first WiFi, and the second WiFi as a wireless network to the packet scheduler (step S40).

In step S40, the sensitivity of each wireless network is analyzed to maximize the transmission success rate and reduce the resources of the network camera and the streaming server, and the optimal packet is scheduled according to the signal-to-noise ratio sensitivity.

To this end, by analyzing the signal-to-noise ratio of each wireless network, at least one wireless network among the LTE, the first WiFi, and the second WiFi is added to the packet scheduler as a wireless network.

Specifically, if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to the first threshold, only the first Wi-Fi is added as a wireless network to the packet scheduler, and if the signal-to-noise ratio of the second Wi-Fi is greater than or equal to the first threshold, the second Wi-Fi Only add it as a wireless network to the packet scheduler.

On the other hand, at least one of the first Wi-Fi and the second Wi-Fi and the LTE with a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi that is less than the first threshold and greater than or equal to the second threshold are transmitted to the packet scheduler as a wireless network. add

In other words, if the signal-to-noise ratio of the first Wi-Fi is less than the first threshold and greater than or equal to the second threshold, the first Wi-Fi and the LTE are added to the packet scheduler as wireless networks.

Also, as in the above, if the signal-to-noise ratio of the second Wi-Fi is less than the first threshold and greater than or equal to the second threshold, the second Wi-Fi and the LTE are added to the packet scheduler as wireless networks.

If the signal-to-noise ratios of the first Wi-Fi and the second Wi-Fi are both less than the first threshold and greater than or equal to the second threshold, the first Wi-Fi and the second Wi-Fi and the LTE are both sent to the packet scheduler as a wireless network is added to

Hereinafter, packet scheduling in step S40 will be described with reference to FIG. 6 . Here, the first Wi-Fi will be described as a Wifi 24GHz wireless network supporting a bandwidth of 24GHz, and the second Wi-Fi will be described as a Wifi 50GHz wireless network supporting a bandwidth of 50GHz.

In addition, it will be described as an example that the first threshold value is 50 dBm and the second threshold value is 20 dBm.

6, the signal-to-noise ratio of each wireless network is analyzed (step S41), and it is checked whether the signal-to-noise ratio of Wifi 24GHz is 50dBm or more (step S42) If the signal-to-noise ratio of Wifi24GHz is 50dBm or more, only Wifi 24GHz is a packet scheduler to the wireless network. (Step S43)

On the other hand, when the signal-to-noise ratio of Wifi 24GHz is less than 50dBm, it is checked whether the signal-to-noise ratio of Wifi 50GHz is 50dBm or higher (step S44). (Step S45)

On the other hand, when the signal-to-noise ratio of the Wifi 50GHz is less than 50dBm, it is checked whether the signal-to-noise ratio of the Wifi 24GHz is 20dBm or more (step S51). is added (step S52), and proceeds to the next step.

When the signal-to-noise ratio of the Wifi 24GHz is less than 20dBm, it is checked whether the signal-to-noise ratio of the Wifi 50GHz is 20dBm or more (step S53) If the signal-to-noise ratio of the Wifi 50GHz is 20dBm or more, the Wifi 50GHz is added to the packet scheduler as a wireless network (Step S54), and proceeds to the next step.

Thereafter, the LTE is added as a wireless network to the packet scheduler (step S55).

Finally, it is checked whether there are a plurality of wireless networks added (step S56), and if not, the video data packet is directly transmitted to the wireless network added to the packet scheduler (step S90).

On the other hand, if there are a plurality of finally added wireless networks (step S56-Yes), a video data packet to be transmitted for each added wireless network is determined (step S70) and the next step is performed.

Thereafter, the determined video data packet is transmitted to the wireless network added to the packet scheduler (step S90).

In step S70, when there are a plurality of wireless networks added in the packet scheduling step (step S40), it is possible to determine an image data packet to be transmitted for each added wireless network.

To this end, when the added wireless network is one of the first Wi-Fi and the second Wi-Fi and the LTE, a packet number selected according to a preset condition among all the image data packets generated by the packetizing unit 120 is included. image data packets to be transmitted to the LTE are determined as image data packets to be transmitted through the LTE, and the remaining image data packets are determined as image data packets to be transmitted through the first Wi-Fi or the second Wi-Fi.

The criterion for determining the image data packet to be transmitted is to determine the picture or packet that must be transmitted by assigning importance according to the location of the picture or packet based on the purpose of collecting the image data, the type of image data, etc., or to be included in one picture It decides the video data packet to be transmitted by distributing the total number of packets to be transmitted.

This is to reduce the time required to reconstruct the image data packet by transmitting the image data packet so that the image data packet is not duplicated, unlike reconstructing the image data packet according to whether the image data packet is duplicated. For example, the network state and setting of the user terminal 50 Accordingly, it may be implemented when it is desired to receive low-resolution image data without delay when receiving image data.

And when the added wireless network is the first Wi-Fi, the second Wi-Fi, and the LTE, an image data packet including a picture number selected according to a preset condition among all image data packets is transmitted to the LTE as an image data packet. is determined, and an image data packet including a packet number selected according to a preset condition among the image data packets not including the selected picture number is determined as the image data packet to be transmitted to the first Wi-Fi, and the remaining image data packets are 2 It can be determined by the image data packet to be transmitted over Wi-Fi.

Meanwhile, in another embodiment of the present invention, when the added wireless network is the LTE, the first Wi-Fi, and the second Wi-Fi, and the importance of a frame indicated by a picture number is set differently, the entire data packet is transmitted to the LTE It is determined as an image data packet, and the image data packet to be transmitted through the first Wi-Fi and the second Wi-Fi may be determined as an image data packet to transmit only an image data packet including a picture number having high importance.

The video data transmitted from the IP-based mobile camera is received according to the network and stored in the database according to the arrival time in the receiving buffer.

The streaming server that has received this stores the transmitted video data packet in the database according to the arrival time. Since the same picture number and packet number in the video data stored in the database represent the same video data in the video stream, the video data with the least reception delay is selected from the video data repeatedly received by multiple transmission, and the unselected video data is discarded. .

The selected video data is collected and reconstructed into a new video stream and transmitted to the user terminal. In order to reduce the network bandwidth when reconstructing the video stream, the IP-based mobile camera unit removes the multiple division and packet number used for multiplexing during packetizing so that more video data can be transmitted.

Accordingly, it provides stable wired/wireless communication connection using dual transmission of LTE and Wifi, and can support customers to use stable video in real time through cloud-based streaming service. In particular, it is possible to receive stable images even in low data band fluctuations in LTE (private wireless network).

As such, the multi-transmission streaming method for the image stabilization service may be implemented as an application or implemented in the form of program instructions that may be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

The program instructions recorded on the computer-readable recording medium are specially designed and configured for the present invention, and may be known and available to those skilled in the computer software field.

Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

Although described above with reference to the embodiments, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below You will understand.

According to the present invention, the service solution according to the present invention can be customized and sold in various markets such as vehicles, ships, military, aviation, ports, forest fire monitoring, and building monitoring by customizing the platform for companies and institutions.

In addition, by utilizing the technology of the present invention, visibility in new markets such as mobile vehicles (unmanned aerial vehicles, ships, special vehicles) can be secured, thereby replacing the market using digital cameras or camcorders, and cloud-type systems using mobile communication networks It is possible to create a new market that can cooperatively control the situation in the field in real time.

10: multi-transmission streaming device 110: encoding unit
120: packetizing unit 130: multi-wireless modem
140: packet scheduler 150: packet decision unit
160: packet multi-transmission unit 30: streaming server
300: cloud platform 310: multi-packet receiver
320: database 330: packet reconstruction module
350: packet transmission unit
360: mobile/terminal connection status management
370: Log management and simultaneous access management
380: OS such as Linux or Windows
50: user terminal 510: wireless modem
530: packet receiving unit 550: decoding unit

Claims (20)

encoding the image data received from the image sensor;
a packetizing step of generating an image data packet in one transmission unit including multiple divisions, a picture number, a packet number, and data to transmit the encoded byte stream;
configuring a network module to communicate simultaneously using a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet;
a packet scheduling step of analyzing the signal-to-noise ratio of each wireless network and adding at least one wireless network among the LTE, the first WiFi, and the second WiFi as a wireless network to a packet scheduler;
determining an image data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and
transmitting the video data packet to a wireless network added to the packet scheduler;
The picture number displays one frame of an image encoded at 30 frames per second as one picture and includes number information for distinguishing each picture,
The packet number includes packet number information for classifying each packet divided into n packets according to the size of one picture,
The step of determining the image data packet to be transmitted for each added wireless network comprises:
When the added wireless network is one of the first Wi-Fi and the second Wi-Fi and the LTE,
An image data packet including a packet number selected according to a preset condition among all image data packets is determined as the image data packet to be transmitted to the LTE, and the remaining image data packets are image data packets to be transmitted through the first Wi-Fi or the second Wi-Fi. to decide,
The selection of the packet number according to the preset condition comprises:
determining the packet number to be transmitted by assigning importance to each position of the packet in the picture based on the purpose of collecting the image data and the type of image data;
When the added wireless network is the first Wi-Fi, the second Wi-Fi and the LTE,
determining an image data packet including a picture number selected according to a preset condition among all image data packets as the image data packet to be transmitted to the LTE;
determining an image data packet including a preset packet number among the remaining image data packets as an image data packet to be transmitted to a Wi-Fi having a higher priority among the first Wi-Fi and the second Wi-Fi;
The remaining image data packets are determined as image data packets to be transmitted to the Wi-Fi with the lower priority,
The selection of the picture number according to the preset condition is determined according to the order of the picture based on the purpose of collecting the image data and the type of image data,
The video data packet including a preset packet number among the remaining video data packets is a packet including a packet number corresponding to a matrix specified according to a predetermined rule among video data packets including a picture number excluding the selected picture number. Characterized in, a method for streaming multiple transmissions of an image.
According to claim 1,
The packet scheduling step of adding the at least one wireless network to the packet scheduler as a wireless network includes:
analyzing the signal-to-noise ratio of each wireless network;
adding only the first Wi-Fi to the packet scheduler as a wireless network when the signal-to-noise ratio of the first Wi-Fi is equal to or greater than a first threshold;
adding only the second Wi-Fi to the packet scheduler as a wireless network when the signal-to-noise ratio of the second Wi-Fi is equal to or greater than the first threshold;
adding the first Wi-Fi and the LTE as a wireless network to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a second threshold and less than a first threshold; and
When the signal-to-noise ratio of the second Wi-Fi is greater than or equal to the second threshold and less than the first threshold, a packet scheduling step of adding the second Wi-Fi and the LTE to the packet scheduler as a wireless network. Way.
delete delete According to claim 1,
The priority is
Multiple transmission streaming method of video, characterized in that the transmission rate, saturation and signal interference are set in order.
According to claim 1,
The multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, multiple transmission streaming method of an image.
According to claim 1,
The first Wi-Fi supports a bandwidth of 24 GHz, and the second Wi-Fi supports a bandwidth of 50 GHz, Multiple transmission streaming method of an image. 3. The method of claim 2,
The first threshold value is 50 dBm, and the second threshold value is 20 dBm.
According to claim 1,
Encoding the image data received from the image sensor,
A multi-transmission streaming method of video encoding the video data input from the sensor in the H264 codec format.
[Claim 10] A computer-readable recording medium in which computer programming is recorded for performing the method of multiple transmission streaming of an image according to any one of claims 1, 2, and 5 to 9.
As a multi-transmission streaming device of an image provided in an IP-based mobile,
an encoding unit for encoding the image data received from the image sensor;
a packetizing unit for generating an image data packet in one transmission unit including multiple divisions, picture numbers, packet numbers, and data to transmit the encoded byte stream;
a multiple wireless modem supporting a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet;
By analyzing the signal-to-noise ratio of each wireless network, if the signal-to-noise ratio of the first WiFi is equal to or greater than a first threshold, only the first WiFi is added as a wireless network to the packet scheduler, and the signal-to-noise ratio of the second WiFi is the first If it is equal to or greater than a threshold, only the second Wi-Fi is added to the packet scheduler as a wireless network, and the first Wi-Fi and the second Wi-Fi having a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi equal to or greater than a second threshold and less than a first threshold. a packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network;
a packet determiner configured to determine a video data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and
and a packet multiplexing unit for transmitting the image data packet to the wireless network added to the packet scheduler,
The picture number displays one frame of an image encoded at 30 frames per second as one picture and includes number information for distinguishing each picture,
The packet number includes packet number information for classifying each packet divided into n packets according to the size of one picture,
The packet determination unit,
When the wireless network added in the packet scheduler is one of the first Wi-Fi and the second Wi-Fi and the LTE,
An image data packet including a packet number selected according to a preset condition among all image data packets is determined as the image data packet to be transmitted to the LTE, and the remaining image data packets are image data packets to be transmitted through the first Wi-Fi or the second Wi-Fi. to decide,
The selection of the packet number according to the preset condition comprises:
Based on the purpose of collecting the image data and the type of image data, importance is given according to the location of the packet in the picture to determine the packet number that must be transmitted as essential;
When the wireless networks added in the packet scheduler are the first Wi-Fi, the second Wi-Fi and the LTE,
determining an image data packet including a picture number selected according to a preset condition among all image data packets as the image data packet to be transmitted to the LTE;
Determining an image data packet including a preset packet number among the remaining image data packets as an image data packet to be transmitted to a higher-priority Wi-Fi among the first Wi-Fi and the second Wi-Fi;
The selection of the picture number according to the preset condition is determined according to the order in the picture based on the purpose of collecting the image data and the type of image data,
A video data packet including a preset packet number among the remaining video data packets includes packet numbers that are even columns in odd rows and odd columns in even rows among video data packets including picture numbers excluding the selected picture number. A multi-transmission streaming device of the video, characterized in that.
delete delete 12. The method of claim 11,
The priority is
A multi-transmission streaming device for video, characterized in that the transmission rate, saturation, and signal interference are set to be high in the order.
12. The method of claim 11,
The multiple classification includes information for classifying wireless networks of the LTE, the first Wi-Fi, and the second Wi-Fi, multiple transmission streaming apparatus of an image.
12. The method of claim 11,
The first Wi-Fi supports a bandwidth of 24 GHz, and the second Wi-Fi supports a bandwidth of 50 GHz, a multi-transmission streaming device of an image. 12. The method of claim 11,
The first threshold value is 50 dBm, and the second threshold value is 20 dBm, multiple transmission streaming apparatus of an image.
12. The method of claim 11,
The encoding unit encodes the image data input from the sensor in an H264 codec format, a multi-transmission streaming device of an image.
A system comprising: a multi-transmission streaming device for an image provided in an IP-based mobile device; and a streaming server for receiving an image data packet transmitted from the multi-transmission streaming device, the multi-transmission streaming device for the video,
an encoding unit for encoding the image data received from the image sensor;
a packetizing unit for generating an image data packet in one transmission unit including multiple divisions, picture numbers, packet numbers, and data to transmit the encoded byte stream;
a multiple wireless modem supporting a wireless network including a first Wi-Fi and a second Wi-Fi having different bandwidths from LTE for multiple transmission of the image data packet;
By analyzing the signal-to-noise ratio of each wireless network, if the signal-to-noise ratio of the first WiFi is equal to or greater than a first threshold, only the first WiFi is added as a wireless network to the packet scheduler, and the signal-to-noise ratio of the second WiFi is the first If it is equal to or greater than a threshold, only the second Wi-Fi is added to the packet scheduler as a wireless network, and the first Wi-Fi and the second Wi-Fi having a signal-to-noise ratio of the first Wi-Fi and the second Wi-Fi equal to or greater than a second threshold and less than a first threshold. a packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network;
a packet determiner configured to determine a video data packet to be transmitted for each added wireless network when a plurality of wireless networks are added to the packet scheduler; and
and a packet multiplexing unit for transmitting the image data packet to the wireless network added to the packet scheduler,
The picture number displays one frame of an image encoded at 30 frames per second as one picture and includes number information for distinguishing each picture,
The packet number includes packet number information for classifying each packet divided into n packets according to the size of one picture,
The packet determination unit,
When the wireless network added in the packet scheduler is one of the first Wi-Fi and the second Wi-Fi and the LTE,
An image data packet including a packet number selected according to a preset condition among all image data packets is determined as the image data packet to be transmitted to the LTE, and the remaining image data packets are image data packets to be transmitted through the first Wi-Fi or the second Wi-Fi. to decide,
The selection of the packet number according to the preset condition comprises:
Based on the purpose of collecting the image data and the type of image data, importance is given according to the location of the packet in the picture to determine the packet number that must be transmitted as essential;
When the wireless networks added in the packet scheduler are the first Wi-Fi, the second Wi-Fi and the LTE,
determining an image data packet including a picture number selected according to a preset condition among all image data packets as the image data packet to be transmitted to the LTE;
Determining an image data packet including a preset packet number among the remaining image data packets as an image data packet to be transmitted to a higher-priority Wi-Fi among the first Wi-Fi and the second Wi-Fi;
The selection of the picture number according to the preset condition is determined according to the order in the picture based on the purpose of collecting the image data and the type of image data,
A video data packet including a preset packet number among the remaining video data packets includes packet numbers that are even columns in odd rows and odd columns in even rows among video data packets including picture numbers excluding the selected picture number. A system, characterized in that
The method of claim 19, wherein the streaming server,
a multi-packet receiver for storing the video data packets transmitted by the multi-transmission streaming in a database according to arrival times;
Packet reconstruction module that selects the image data stored in the database first according to the arrival priority time and discards the unselected image data when the multiple classification values are different for image data having the same picture number and packet number in the stored image multiplex data ; and
When transmitting the reconstructed packet to the user terminal, the system characterized in that it comprises a packet transmission unit for transmitting the image data by removing the multi-classification and packet number fields.

Images