KR101670767B1 - Method for duplicated streaming transmission to provide image stabilization service, recording medium, device and sysytem for performing the method - Google Patents

Abstract

A method for duplicated streaming transmission to provide an image stabilization service comprises the steps of: encoding image data received from an image sensor; generating image data packets including duplication, a picture number, a packet number, and data in a single transmission unit in order to transmit encoded byte stream; configuring a network module so as to perform communications using wireless networks including LTE, first Wi-Fi, and second Wi-Fi having different bandwidths for duplicated transmission of the image data packets; analyzing a signal-to-noise ratio of the respective wireless networks, and adding at least one wireless network of the LTE, the first Wi-Fi and the second Wi-Fi to a packet scheduler; and transmitting the image data packets through the wireless network added to the packet scheduler. Accordingly, a stable image can be received from a mobile camera.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a duplicated transmission streaming method for an image stabilization service, a recording medium, an apparatus and a system for performing the same,

The present invention relates to a multi-network-based streaming method and system for wireless transmission of video and object Internet images of a moving object, and more particularly, to an IP-based streaming method and system for multi- In order to provide a stable real-time transmission service of a camera image, a streaming service for transmitting the captured image to a multi-channel wireless Internet network.

In recent years, convergence has been taking place not only in the domestic market but also in the overseas market. Text-based services are rapidly changing to video based on the emergence of smart phones. In this environment, existing CCTV video surveillance is mainly a limited market where the inside of the building is monitored centrally or remotely by PC, however, the demand for real-time monitoring of camera images through mobile bodies is increasing.

In addition, due to the rapid development of the wireless mobile communication network infrastructure and the development of smart phones and tablet PCs, the demand for real-time simultaneous connection is increasing, and the ultra-small image stabilization processing of the Internet is required through the development of high performance embedded CPUs.

For example, there is a growing demand for automatically generating stereoscopic images based on images acquired from video cameras and smart phones, or reproducing realistic virtual reality (VR) images viewed at different points in a new path. .

Particularly, there is an increasing demand for producing a video that looks at an object at a point in time that has not been experienced in the past by utilizing a drone. As a result, studies related to aerial photography using a drone have been actively conducted. In addition, images are taken in various fields for vehicles, ships, military, air, harbor, forest fire monitoring, and building surveillance.

Accordingly, it is necessary to mount video equipment on a moving object to transmit video information in real time without any image stickiness, and to perform real-time control and scouting.

KR 10-2016-0072971 A KR 10-2016-0070922 A

Accordingly, it is an object of the present invention to provide a redundant transmission streaming method for an image stabilization service.

It is another object of the present invention to provide a recording medium on which a computer program for performing a redundant transmission streaming method for the image stabilization service is recorded.

It is still another object of the present invention to provide an apparatus for performing a redundant transmission streaming method for the image stabilization service.

Still another object of the present invention is to provide a system including a redundant transmission streaming device for the image stabilization service.

According to another aspect of the present invention, there is provided a duplicated transmission streaming method for an image stabilization service, comprising: encoding image data received from an image sensor; A packetizing step of generating an image data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data in order to transmit an encoded byte stream; Configuring a network module to simultaneously communicate using LTE and a wireless network including a first WiFi and a second WiFi having different bandwidths for dual transmission of the video data packet; A packet scheduling step of analyzing a signal-to-noise ratio of each wireless network and adding at least one of the LTE, the first Wi-Fi and the second Wi-Fi to a packet scheduler as a wireless network; And transmitting the video data packet to a wireless network added to the packet scheduler.

In an embodiment of the present invention, the packet scheduling step of adding the at least one wireless network to a packet scheduler as a wireless network comprises: analyzing a signal to noise ratio of each wireless network; Adding the first Wi-Fi only to the packet scheduler as a wireless network if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value; Adding the second Wi-Fi only to the packet scheduler as a wireless network if the signal-to-noise ratio of the second Wi-Fi is greater than or equal to the first threshold; Adding the first Wi-Fi and the LTE to the packet scheduler in a wireless network if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a second threshold (less than the first threshold); And a packet scheduling step of adding the second WiFi and the LTE to the packet scheduler as a wireless network if the signal-to-noise ratio of the second WiFi is equal to or greater than the second threshold value.

In the embodiment of the present invention, the packetizing step of generating the video data packet includes transmitting the video data in a transmission unit of 1500 bytes plus 40 bytes of header of TCP / IP and 1460 bytes of payload, Packets can be generated.

In the embodiment of the present invention, 1460 bytes of the payload may include a 1-byte redundancy division, a 2-byte picture number, a 2-byte packet number, and 1455-byte data.

In the exemplary embodiment of the present invention, the redundancy classification includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi, and the picture number includes one frame of an image encoded at 30 frames per second And includes number information for identifying each picture. The packet number may include packet number information for identifying each packet divided into n packets according to a size of one picture.

In an embodiment of the present invention, the first Wi-Fi supports a bandwidth of 2.4 GHz, and the second Wi-Fi may support a bandwidth of 5.0 GHz.

In an embodiment of the present invention, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

In the embodiment of the present invention, the encoding unit may encode the image data input from the sensor in the H.264 codec format.

According to another aspect of the present invention, there is provided a computer-readable storage medium storing a computer program for performing a redundant transmission streaming method for an image stabilization service.

According to another aspect of the present invention, there is provided a dual stream transmission streaming apparatus for an image stabilization service provided in an IP-based moving object, comprising: an encoding unit for encoding image data received from an image sensor; A packet tacking unit for generating a video data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data to transmit an encoded byte stream; A multiple wireless modem supporting a wireless network including a first WiFi and a second WiFi with different bandwidths from LTE for dual transmission of the video data packet; Wherein the signal-to-noise ratio of each of the wireless networks is analyzed to add only the first Wi-Fi to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value, Wherein the first WiFi and the second WiFi have a signal-to-noise ratio greater than or equal to a second threshold value (less than the first threshold) And a packet scheduler that adds at least one of the second Wi-Fi and the LTE to the packet scheduler in a wireless network; And a packet duplication transmitting unit for transmitting the video data packet to a wireless network added to the packet scheduler.

In an exemplary embodiment of the present invention, the transmission unit of the video data packet may be 1500 bytes including 40 bytes of the TCP / IP header and 1460 bytes of the payload.

In the embodiment of the present invention, 1460 bytes of the payload may include a 1-byte redundancy division, a 2-byte picture number, a 2-byte packet number, and 1455-byte data.

In the exemplary embodiment of the present invention, the redundancy classification includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi, and the picture number includes one frame of an image encoded at 30 frames per second And includes number information for identifying each picture. The packet number may include packet number information for identifying each packet divided into n packets according to a size of one picture.

In an embodiment of the present invention, the first Wi-Fi supports a bandwidth of 2.4 GHz, and the second Wi-Fi may support a bandwidth of 5.0 GHz.

In an embodiment of the present invention, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

In the embodiment of the present invention, the encoding unit may encode the image data input from the sensor in the H.264 codec format.

According to another aspect of the present invention, there is provided a system for providing an IP-based moving object, comprising: a dual stream transmission streaming device for an image stabilization service and an image data packet received from the dual stream transmission streaming device; The dual stream transmission apparatus includes: an encoding unit configured to encode image data received from an image sensor; A packet tacking unit for generating a video data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data to transmit an encoded byte stream; A multiple wireless modem supporting a wireless network including a first WiFi and a second WiFi with different bandwidths from LTE for dual transmission of the video data packet; Wherein the signal-to-noise ratio of each of the wireless networks is analyzed to add only the first Wi-Fi to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value, Wherein the first WiFi and the second WiFi have a signal-to-noise ratio greater than or equal to a second threshold value (less than the first threshold) And a packet scheduler that adds at least one of the second Wi-Fi and the LTE to the packet scheduler in a wireless network; And a packet duplication transmitting unit for transmitting the video data packet to a wireless network added to the packet scheduler.

In an exemplary embodiment of the present invention, the streaming server may include a duplicated packet receiver for storing the received image data packet in the database according to the arrival time of the image data packet transmitted by the duplicated transmission streaming; A packet reconstruction module for selecting image data stored in the database according to the arrival priority time and discarding unselected image data when the duplication discrimination value is different for the image data having the same picture number and packet number in the stored image duplication data, ; And a packet transmission unit for transmitting the reconstructed packet to the user terminal by removing the redundancy classification and the packet number field and transmitting the image data.

According to the duplicated transmission streaming method for the image stabilization service, a stable wired / wireless communication connection is provided by collaborating with LTE / LTE-m and WiFi network operators, which are next-generation wireless communication services in commercial service, and a cloud-based streaming service To enable customers to use stable images in real time. In particular, LTE (private wireless network) can receive stable images even in low data band fluctuations.

1 is a conceptual diagram of a redundant transmission streaming system for an image stabilization service according to an embodiment of the present invention.
2 is a block diagram of a redundant transmission streaming system for an image stabilization service according to an exemplary embodiment of the present invention.
3 is a conceptual diagram of a TCP / IP segment and a transmission packet for video data duplication transmission according to the present invention.
4 is a conceptual diagram of packet reconstruction based on packet loss and arrival priority time.
FIG. 5 is a flowchart illustrating a duplicated transmission streaming method for an image stabilization service according to an exemplary embodiment of the present invention.
6 is a detailed flowchart of packet scheduling of FIG.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a conceptual diagram of a redundant transmission streaming system for an image stabilization service according to an embodiment of the present invention. 2 is a block diagram of a redundant transmission streaming system for an image stabilization service according to an exemplary embodiment of the present invention.

A redundant transmission streaming system 1 for a video stabilization service according to the present invention includes a redundant transmission streaming apparatus 10 for a video stabilization service, a streaming server 30, and a user terminal 50 And stably provides the video data received from the device 10 to the user terminal 50 through the streaming server 30. [

The device 10 may be included in or communicated to an IP based mobile camera that collects image data. For example, the device 10 may be included in video equipment installed in a drones, a vehicle, a vessel, a military, an airplane, a port, or the like.

According to the present invention, in order to provide a stable real-time transmission service of an IP-based camera mounted on a moving object, the captured image is transmitted through LTE and WiFi.

1 and 2, an apparatus 10 according to the present invention includes an encoding unit 110, a packetizing unit 130, a multiple wireless modem 150, a packet scheduler 170, and a packet duplication transmitting unit 190 ).

The apparatus 10 of the present invention may be implemented with software (application) for performing redundant transmission streaming for an image stabilization service and may be implemented by the encoding unit 110, the packet tying unit 130, The configuration of the modem 150, the packet scheduler 170 and the packet duplication transmitter 190 may be controlled by software for performing redundant transmission streaming for the image stabilization service executed in the device 10. [ have.

The device 10 may be a separate terminal or some module of the terminal. The configuration of the encoding unit 110, the packetizing unit 130, the multiple wireless modem 150, the packet scheduler 170, and the packet duplication transmitting unit 190 may be formed by an integrated module, Or more. However, conversely, each configuration may be a separate module.

The device 10 may be mobile or stationary. The device 10 may be in the form of a server or an engine and may be a device, an apparatus, a terminal, a user equipment (UE), a mobile station (MS) a wireless device, a handheld device, and the like.

The device 10 may execute or produce various software based on an operating system (OS), i.e., a system. The operating system is a system program for allowing software to use the hardware of a device. The operating system includes a mobile computer operating system such as Android OS, iOS, Windows Mobile OS, Sea OS, Symbian OS, Blackberry OS, MAC, AIX, and HP-UX.

The encoding unit 110 encodes image data received from an image sensor (for example, a camera). The encoding unit 110 may encode the input image data in a codec format and may encode the image data in the H.264 format, for example.

The packetizing unit 130 generates a video data packet in one transmission unit to transmit the encoded byte stream.

Referring to FIG. 3, the packet-taining unit 130 may set 1500 bytes as one transmission unit by adding 40 bytes of the header of the TCP / IP and 1460 bytes of the payload. The 1460 bytes of the payload include a 1-byte redundant segment, a 2-byte picture number, a 2-byte packet number, and 1455-byte data.

Wherein the redundancy information includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi, wherein the picture number indicates one frame of an image encoded with 30 frames per second as one picture, And the packet number includes packet number information for distinguishing each packet divided into n packets according to a size of one picture.

The multi-wireless modem 150 is connected to the network module 150 so that it can simultaneously communicate with the wireless network including the first Wi-Fi (WiFi) and the second Wi-Fi having different bandwidths from the LTE for dual transmission of the generated video data packet. .

The multi-radio modem 150 configures a network module so that the device 10 can communicate using three wireless networks at the same time.

The first Wi-Fi may support a bandwidth of 2.4 GHz, and the second Wi-Fi may be a wireless network that supports a bandwidth of 5.0 GHz. Wifi 2.4GHz has a wide range (coverage), but it is almost saturated, so there may be signal interference. Wifi 5GHz is faster than Wifi 2.4GHz but has narrow range (coverage), strong linearity, and weakness at corners.

However, this is merely an example and a wireless network that supports different bandwidths as needed, or a wireless network that supports different bandwidths may be added.

The packet scheduler 170 analyzes the sensitivity of each wireless network to maximize the transmission success rate and reduce resources of the network camera and the streaming server, and schedules the optimal packet according to the SNR sensitivity.

To this end, the packet scheduler 170 analyzes the signal-to-noise ratio of each wireless network and adds at least one of the LTE, the first Wi-Fi, and the second Wi-Fi to the packet scheduler as a wireless network.

If the signal-to-noise ratio of the first WiFi is greater than or equal to the first threshold value, only the first Wi-Fi is added to the packet scheduler as a wireless network. If the signal- Only to the packet scheduler as a wireless network.

Meanwhile, at least one of the first WiFi and the second WiFi in which the signal-to-noise ratio of the first WiFi and the second WiFi is equal to or greater than a second threshold (smaller than the first threshold) Add to the network.

In other words, if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a second threshold (less than the first threshold), the first Wi-Fi and the LTE are added to the packet scheduler as a wireless network. Also, if the signal-to-noise ratio of the second Wi-Fi is equal to or greater than the second threshold, the second Wi-Fi and the LTE are added to the packet scheduler as a wireless network.

In this case, if the signal-to-noise ratios of the first WiFi and the second WiFi are both equal to or greater than a second threshold value, both the first WiFi and the second WiFi and the LTE are added to the wireless network by the packet scheduler.

For example, the first threshold value may be 50 dBm, and the second threshold value may be 20 dBm.

The packet duplication transmitting unit 190 transmits the video data packet to the wireless network added to the packet scheduler.

The streaming server 30 includes a cloud platform 300, a duplication packet receiver 310, a packet reconstruction module 330, and a packet transmitter 350, which provide a cloud based simultaneous multiplex transmission platform (LTE, WiFi) . The streaming server 30 may further include a mobile / terminal connection state management 360, a log management and concurrent access management 370, and an OS (operating system) 380 such as Linux or Windows.

The duplication packet receiver 310 stores the received duplication packet in the database 320 according to the arrival time of the image data packet transmitted by the duplicated transmission streaming.

The packet reconstruction module 330 selects the image data stored in the database according to the arrival priority time if the duplication discrimination value is different from the image data having the same picture number and packet number in the stored image duplication data, The image data is discarded.

FIG. 4 is a conceptual diagram of packet reconstruction based on packet loss and arrival priority time.

Referring to FIG. 4, since the same picture number and packet number in the video data stored in the database 320 indicate the same video data in the video stream, the packet reconstruction module 330 can repeatedly receive the video data The image data having the smallest reception delay is selected and the image data not selected is discarded.

Accordingly, the reconstructed packets shown in FIG. 4 are arranged in the order of the picture number and the packet numbers constituting each picture number. Referring to the information of the redundancy division for distinguishing the wireless network, the packets received from the LTE and the Wi- As shown in FIG.

Accordingly, the selected video data is collected and reconstructed into a new video stream and transmitted to the user terminal 50.

When transmitting the reconstructed packet to the user terminal 50, the packet transmitter 350 removes the redundancy classification and the packet number field and transmits the video data.

In order to reduce the network bandwidth, the packet transmission unit 350 removes the redundancy classification and the packet number used for the redundancy in packet timing in the device 10 so that more image data can be transmitted.

The user terminal 50 includes a wireless modem 510, a packet receiving unit 530 and a decoding unit 550 that support a wireless network and can receive images received from the streaming server 30 stably.

As a result, cloud-based streaming services can provide reliable images to customers in real time. In particular, LTE (private wireless network) can receive stable images even in low data band fluctuations.

FIG. 5 is a flowchart illustrating a duplicated transmission streaming method for an image stabilization service according to an exemplary embodiment of the present invention. 6 is a detailed flowchart of packet scheduling of FIG.

The redundant transmission streaming method for the image stabilization service according to the present embodiment can be performed in substantially the same configuration as the apparatus 10 of FIG. Therefore, the same constituent elements as those of the apparatus 10 of FIG. 1 are denoted by the same reference numerals, and repeated description is omitted. In addition, the redundant transmission streaming method for the image stabilization service according to the present embodiment can be executed by software (application) for performing redundant transmission streaming for the image stabilization service.

According to the present invention, in order to provide a stable real-time transmission service of an IP-based camera mounted on a moving object, the captured image is transmitted through LTE and WiFi. The moving body may be a drones, a vehicle, a vessel, a military, an airplane, a port, or the like.

Referring to FIG. 5, a redundant transmission streaming method for an image stabilization service according to the present embodiment encodes image data received from an image sensor (step S10). At this time, the input image data can be encoded in the codec format, for example, in the H.264 format.

In order to transmit the encoded byte stream, packetization is performed to generate a video data packet in one transmission unit including a redundancy segment, a picture number, a packet number and data (step S30).

By adding 40 bytes of header in TCP / IP and 1460 bytes in payload, 1500 bytes can be defined as one transmission unit. The 1460 bytes of the payload include a 1-byte redundant segment, a 2-byte picture number, a 2-byte packet number, and 1455-byte data.

Wherein the redundancy information includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi, wherein the picture number indicates one frame of an image encoded with 30 frames per second as one picture, , And the packet number includes packet number information for distinguishing each packet divided into n packets according to the size of one picture (refer to FIG. 3).

In step S50, a network module is configured to communicate using LTE and a wireless network including a first WiFi and a second WiFi having different bandwidths simultaneously for dual transmission of the generated video data packet.

At this time, a network module is configured so that three wireless networks can be used simultaneously to communicate with each other.

The first Wi-Fi may support a bandwidth of 2.4 GHz, and the second Wi-Fi may be a wireless network that supports a bandwidth of 5.0 GHz. Wifi 2.4GHz has a wide range (coverage), but it is almost saturated, so there may be signal interference. Wifi 5GHz is faster than Wifi 2.4GHz but has narrow range (coverage), strong linearity, and weakness at corners.

However, this is merely an example and a wireless network that supports different bandwidths as needed, or a wireless network that supports different bandwidths may be added.

(Step S70). In step S70, a signal-to-noise ratio of each wireless network is analyzed to add at least one of the LTE, the first WiFi, and the second WiFi to the packet scheduler as a wireless network.

In step S70, the sensitivity of each wireless network is analyzed to maximize the transmission success rate and to reduce the resources of the network camera and the streaming server, and the optimal packet is scheduled according to the signal-to-noise ratio sensitivity.

To this end, the signal-to-noise ratio of each wireless network is analyzed and at least one of the LTE, the first Wi-Fi, and the second Wi-Fi is added to the packet scheduler as a wireless network.

If the signal-to-noise ratio of the first WiFi is greater than or equal to the first threshold value, only the first Wi-Fi is added to the packet scheduler as a wireless network. If the signal- Only to the packet scheduler as a wireless network.

Meanwhile, at least one of the first WiFi and the second WiFi in which the signal-to-noise ratio of the first WiFi and the second WiFi is equal to or greater than a second threshold (smaller than the first threshold) Add to the network.

In other words, if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a second threshold (less than the first threshold), the first Wi-Fi and the LTE are added to the packet scheduler as a wireless network. Also, if the signal-to-noise ratio of the second Wi-Fi is equal to or greater than the second threshold, the second Wi-Fi and the LTE are added to the packet scheduler as a wireless network.

In this case, if the signal-to-noise ratios of the first WiFi and the second WiFi are both equal to or greater than a second threshold value, both the first WiFi and the second WiFi and the LTE are added to the wireless network by the packet scheduler.

Hereinafter, packet scheduling in step S70 will be described with reference to FIG. Here, the first Wi-Fi is a Wifi 2.4 GHz wireless network supporting a bandwidth of 2.4 GHz, and the second Wi-Fi is a Wifi 5.0 GHz wireless network supporting a bandwidth of 5.0 GHz. The first threshold value is 50 dBm, and the second threshold value is 20 dBm.

Referring to FIG. 6, the signal-to-noise ratio of each wireless network is analyzed (step S71), and it is confirmed whether the signal-to-noise ratio of Wifi 2.4 GHz is 50 dBm or more (step S72). When the signal-to-noise ratio of 2.4 GHz is over 50 dBm, only Wifi 2.4 GHz is added to the packet scheduler as a wireless network (step S73).

On the other hand, if the signal-to-noise ratio of the Wifi 2.4GHz is less than 50dBm, it is confirmed whether the signal-to-noise ratio of the Wifi 5.0GHz is 50dBm or more (step S74). When the signal-to-noise ratio of the Wifi 5.0 GHz is 50 dBm or more, only the Wifi 5.0 GHz is added to the packet scheduler as a wireless network (step S75).

On the other hand, if the signal-to-noise ratio of the Wifi 5.0 GHz is less than 50 dBm, it is confirmed whether the signal-to-noise ratio of the Wifi 2.4 GHz is 20 dBm or more (step S81). If the signal-to-noise ratio of the Wifi 2.4 GHz is 20 dBm or more, the Wifi 2.4 GHz is added to the packet scheduler as a wireless network (step S82), and the process proceeds to the next step.

If the signal-to-noise ratio of the Wifi 2.4GHz is less than 20dBm, it is checked whether the signal-to-noise ratio of the Wifi 5.0GHz is 20dBm or more (step S83). When the signal-to-noise ratio of the Wifi 5.0 GHz is 20 dBm or more, the Wifi 5.0 GHz is added to the packet scheduler as a wireless network (step S84), and the process proceeds to the next step.

Thereafter, the LTE is added to the packet scheduler as a wireless network (step S85). Accordingly, it is possible to duplicate video data through LTE and Wifi to provide stable streaming transmission.

The video data packet is transmitted to the wireless network added to the packet scheduler in the above manner (step S90).

Based on the network, each image data transmitted from the IP based mobile body camera is received and stored in the database according to the time of arrival in the receiving buffer.

The streaming server receives the video data packet and stores it in the database according to the arrival time. Since the same picture number and packet number in the video data stored in the database represent the same video data in the video stream, the video data having the smallest reception delay is selected from the video data repeatedly received by the redundant transmission and the video data not selected is discarded .

The selected video data is collected and reconstructed into a new video stream and transmitted to the user terminal. In order to reduce the network bandwidth during the reconstruction of the video stream, it is possible to transmit more image data by removing the redundancy classification and the packet number which were used in the packetizing at the IP based mobile camera unit.

As a result, it can provide stable wired / wireless communication connection using dual transmission of LTE and Wifi, and can provide stable image to customers in real time through cloud-based streaming service. In particular, LTE (private wireless network) can receive stable images even in low data band fluctuations.

Such a duplicated transmission streaming method for an image stabilization service can be implemented in an application or implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer-readable recording medium may be ones that are specially designed and configured for the present invention and are known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims. You will understand.

The present invention can customize and sell a service solution according to the present invention in various markets such as a vehicle, a ship, a military, an air, a port, a forest fire,

Further, by utilizing the technique of the present invention, it is possible to replace the market using a conventional digital camera or camcorder by securing visibility even in a moving object (unmanned aerial vehicle, ship, special vehicle) as a new market, It is possible to create a new market in which the situation of the field can be controlled in real time.

1: Redundant transmission streaming system
10: Redundant transmission streaming device
110: encoding section
130: Packet tie-
150: Multiple wireless modems
170: Packet Scheduler
190: Packet duplicated transmission unit
30: streaming server
300: Cloud platform
310: duplication packet receiver
320: Database
330: Packet Reconstruction Module
350: Packet transmission unit
360: Status management of mobile / terminal connection
370: Log Management and Concurrent Access Management
380: OS such as Linux or Windows
50: User terminal
510: Wireless Modem
530: Packet receiver
550:

Claims (18)

Encoding image data received from an image sensor;
A packetizing step of generating an image data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data in order to transmit an encoded byte stream;
Configuring a network module to simultaneously communicate using LTE and a wireless network including a first WiFi and a second WiFi having different bandwidths for dual transmission of the video data packet;
A packet scheduling step of analyzing a signal-to-noise ratio of each wireless network and adding at least one of the LTE, the first Wi-Fi and the second Wi-Fi to a packet scheduler as a wireless network; And
And transmitting the video data packet to a wireless network added to the packet scheduler.
2. The method of claim 1, wherein the packet scheduling step of adding the at least one wireless network to a packet scheduler as a wireless network comprises:
Analyzing a signal-to-noise ratio of each wireless network;
Adding the first Wi-Fi only to the packet scheduler as a wireless network if the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value;
Adding the second Wi-Fi only to the packet scheduler as a wireless network if the signal-to-noise ratio of the second Wi-Fi is greater than or equal to the first threshold;
Adding the first WiFi and the LTE to the packet scheduler as a wireless network if the signal-to-noise ratio of the first WiFi is greater than a second threshold and less than a first threshold; And
And adding the second WiFi and the LTE to the packet scheduler as a wireless network if the signal-to-noise ratio of the second WiFi is greater than or equal to the second threshold value.
2. The method of claim 1, wherein the packetizing step of generating the video data packet comprises:
Wherein the image data packet is generated in a transmission unit of 1500 bytes plus 40 bytes of header of TCP / IP and 1460 bytes of payload.
The method of claim 3,
Wherein the 1460 bytes of the payload include a 1-byte redundancy segment, a 2-byte picture number, a 2-byte packet number, and 1455 bytes of data.
The method according to claim 1,
Wherein the redundancy classification includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi,
Wherein the picture number indicates one frame of an image encoded with 30 frames per second as one picture and includes number information for identifying each picture,
Wherein the packet number includes packet number information for identifying each packet divided into n packets according to a size of one picture.
The method according to claim 1,
Wherein the first Wi-Fi supports a bandwidth of 2.4 GHz and the second Wi-Fi supports a bandwidth of 5.0 GHz.
3. The method of claim 2,
Wherein the first threshold value is 50 dBm and the second threshold value is 20 dBm.
The method according to claim 1,
The encoding of the image data received from the image sensor,
A duplex transmission streaming method for an image stabilization service that encodes video data input from a sensor into an H.264 codec format.
A computer-readable recording medium on which a computer program is recorded, for performing a redundant transmission streaming method for an image stabilization service according to any one of claims 1 to 8.
A dual stream transmission streaming apparatus for an image stabilization service provided in an IP-based moving object,
An encoding unit for encoding image data received from an image sensor;
A packet tacking unit for generating a video data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data to transmit an encoded byte stream;
A multiple wireless modem supporting a wireless network including a first WiFi and a second WiFi with different bandwidths from LTE for dual transmission of the video data packet;
Wherein the signal-to-noise ratio of each of the wireless networks is analyzed to add only the first Wi-Fi to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value, Wherein the first Wi-Fi and the second Wi-Fi add the second Wi-Fi only to the packet scheduler if the first Wi-Fi signal and the second Wi-Fi signal are equal to or greater than a threshold value, A packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network; And
And a packet duplication transmitting unit for transmitting the video data packet to a wireless network added to the packet scheduler.
11. The method of claim 10,
Wherein the transmission unit of the video data packet is 1500 bytes including 40 bytes of header of TCP / IP and 1460 bytes of payload.
12. The method of claim 11,
Wherein the 1460 bytes of the payload include a 1-byte redundant segment, a 2-byte picture number, a 2-byte packet number, and 1455-byte data.
11. The method of claim 10,
Wherein the redundancy classification includes information for distinguishing the wireless network of the LTE, the first WiFi, and the second WiFi,
Wherein the picture number indicates one frame of an image encoded with 30 frames per second as one picture and includes number information for identifying each picture,
Wherein the packet number includes packet number information for distinguishing each packet divided into n packets according to a size of one picture.
11. The method of claim 10,
Wherein the first Wi-Fi supports a bandwidth of 2.4 GHz and the second Wi-Fi supports a bandwidth of 5.0 GHz.
11. The method of claim 10,
Wherein the first threshold value is 50 dBm and the second threshold value is 20 dBm.
11. The method of claim 10,
Wherein the encoding unit encodes the image data input from the sensor into the H.264 codec format.
A dual stream transmission streaming apparatus for an image stabilization service provided in an IP based moving object and a streaming server for receiving a video data packet received from the dual stream transmission streaming apparatus,
An encoding unit for encoding image data received from an image sensor;
A packet tacking unit for generating a video data packet in one transmission unit including a redundancy classification, a picture number, a packet number and data to transmit an encoded byte stream;
A multiple wireless modem supporting a wireless network including a first WiFi and a second WiFi with different bandwidths from LTE for dual transmission of the video data packet;
Wherein the signal-to-noise ratio of each of the wireless networks is analyzed to add only the first Wi-Fi to the packet scheduler when the signal-to-noise ratio of the first Wi-Fi is greater than or equal to a first threshold value, Wherein the first Wi-Fi and the second Wi-Fi add the second Wi-Fi only to the packet scheduler if the first Wi-Fi signal and the second Wi-Fi signal are equal to or greater than a threshold value, A packet scheduler for adding at least one of Wi-Fi and the LTE to the packet scheduler as a wireless network; And
And a packet duplication transmitter for transmitting the video data packet to a wireless network added to the packet scheduler.
18. The streaming server of claim 17,
A duplicated packet receiving unit for storing the duplicated transmission stream in a database according to the arrival time of the transmitted image data packet;
A packet reconstruction module for selecting image data stored in the database according to the arrival priority time and discarding unselected image data when the duplication discrimination value is different for the image data having the same picture number and packet number in the stored image duplication data, ; And
And a packet transmission unit for transmitting the reconstructed packet to the user terminal by removing the redundancy classification and the packet number field to transmit the image data.

Images