KR20220146068A - Intelligent Video Streaming Transmission Control Device - Google Patents

Abstract

The present invention relates to an LET-based intelligent video streaming transmission control device and, more particularly, to an LET-based intelligent video streaming transmission control device that is connected to an IP camera and intelligently streams the video of the IP camera to minimize data transmission and reduce traffic load and data communication cost.

Description

Intelligent Video Streaming Transmission Control Device

The present invention relates to an intelligent video streaming transmission control device, and more particularly, it is connected to an IP camera, and by intelligently streaming the video of the IP camera, the data transmission amount can be minimized to reduce the traffic load and data communication cost. It relates to an intelligent video streaming transmission control device.

An IP camera is a digital video camera that can send and receive data over a computer network and the Internet, and has a fixed IP address so it can deliver streaming to other Internet-connected devices.

In the conventional streaming system, because the system is designed on the assumption that a stable network and sufficient bandwidth are secured without considering the limited resource environment of the wireless state, motion detection simply displays an alarm to the user. It is used only for the purpose of precision, and the full streaming method that transmits all of the input stream data regardless of motion detection is used.

However, the conventional streaming system has a high network data usage, so in a wireless system where the network usage is limited, there is a limit to the user's available data. There was a problem in that the cost burden for data use greatly increased.

In addition, in the case of the full-streaming method, power consumption increases according to the operation of the data processing device, thereby reducing energy efficiency.

As devised to solve the above problems, an object of the present invention is to wirelessly transmit video data of an IP camera in real-time streaming by being connected to an IP camera, but by intelligently controlling the streaming method to efficiently manage traffic to communicate data It is to provide an intelligent video streaming transmission control device that can dramatically reduce costs.

In order to achieve the above object, the intelligent video streaming transmission control device according to the present invention determines one of the full streaming and partial streaming methods through intelligent analysis according to motion detection of the obtained IP camera video to control streaming. and a streaming controller; It is connected to the IP camera and transmits the video data of the IP camera to the streaming controller, controls the settings of the IP camera, and transmits image frame data through a wireless network according to the streaming method determined through the streaming controller. It is characterized in that it includes a video transmitter and receiver.

The streaming controller includes: a signal processing module for processing the acquired IP camera image; a streaming control module for determining a streaming method by analyzing the signal-processed image data; The streaming control module includes: a state determination module for determining whether an operation is in an active state or an idle state by comparing a reference change value corresponding to a motion detection set by a user when a change value of a frame is input from the signal processing module; a frame separation module for inputting a frame to the active processing module when the state determined by the state determination module is an active state, and for inputting a frame to the idle processing module when in an idle state; an active state streaming module for transmitting the active frame input by the frame separation module in full streaming; It characterized in that it comprises an idle state streaming module for transmitting the idle frame input by the frame separation module in partial streaming.

the streaming control module further includes a frame buffer for temporarily storing a frame when a state is changed; When the state determination module is switched from the idle state to the active state due to motion detection, or the active state is switched from the active state to the idle state due to the release of the motion detection, the frame output from the signal processing module is not changed immediately for a set time. It is stored in the frame buffer, and when the switched state is maintained for a set time, the frame stored in the frame buffer is input to the active state streaming module or the idle state streaming module corresponding to the switched state.

and a port module connected to the IP camera to receive image data from the IP camera and input the received image data to the signal processing module; a setting control module for controlling the setting of the IP camera; It characterized in that it comprises a network module for transmitting through a wireless network according to the streaming method determined from the streaming control module.

The setting control module controls settings necessary for the operation of the IP camera, and controls the setting of a reference change value for motion detection that distinguishes an active state and an idle state for image data obtained from the IP camera; Active mode that controls to always stream in an active state regardless of motion detection, idle mode that controls partial streaming in an idle state regardless of motion detection, and controls streaming by switching between active and idle states according to motion detection It is characterized in that it provides an interface for selecting one operation mode among the mixed modes.

Here, the setting control module is characterized in that it provides an interface that can set the operation mode differently for each time period.

As described above, the intelligent video streaming transmission control device according to the present invention is connected to the IP camera and wirelessly transmits the video data of the IP camera in real-time streaming, but intelligently controls the streaming method to efficiently manage the traffic to reduce the data communication cost. There is an excellent effect that can be dramatically reduced.

And because setting is possible only by connecting to the port of the IP camera, it is easy to install and can be universally applied to IP cameras installed in remote places where monitoring and supervision are difficult.

1 is a system configuration diagram schematically showing an LTE-based intelligent video streaming transmission control apparatus according to a preferred embodiment of the present invention;
FIG. 2 is a detailed block diagram of FIG. 1 .
3 is a reference diagram for setting a transmission period in an idle state according to an embodiment of the present invention.
4 shows that the frame buffer according to a preferred embodiment of the present invention is further included.
5 is a logic flowchart for an idle state and an active state and frame streaming according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

The present invention recognizes an intelligent detect according to the movement of a subject photographed by a camera module during stream transmission, and divides it into an active state and an idle state, and the divided idle state and It is characterized in that it is possible to reduce the amount of data transmission compared to the prior art by dividing the duplicated image and the non-overlapping image according to the active state, blocking the streaming transmission of the repeated image, and delivering only the necessary image to the user.

Here, the motion detection is a criterion for distinguishing between the active state and the idle state, and the motion detection requires full streaming when the frame change value according to the movement of the subject exceeds the threshold value of the reference change value compared with the preset reference change value. It is determined that the motion detection event has occurred and the active state is determined, and when the reference change value is not exceeded, the idle state is determined.

That is, the active state means a state in which full streaming is required because a motion detection event occurs, and in the idle state, no detection event occurs and the image change is insufficient, so full streaming is not required. means

1 is a system configuration diagram schematically illustrating an LTE-based intelligent video streaming transmission control apparatus according to a preferred embodiment of the present invention, and FIG. 2 is a detailed block diagram of FIG.

1 and 2, the intelligent video streaming transmission control device according to the present invention is connected to the streaming controller 10 and the IP camera 30, which determine the streaming method through intelligent analysis of the acquired IP camera image, It may be configured to include a wireless video transmitter/receiver 20 for transmitting the video data of the IP camera to the streaming controller and wirelessly transmitting video frame data according to a streaming method determined through the streaming controller.

The streaming controller 10 plays a role of controlling streaming in full streaming in the active state and partial streaming in the idle state according to motion detection for efficient control of traffic.

In addition, the wireless video transceiver 20 is responsible for transmitting the frame to the manager terminal 40 through the wireless network through the streaming method determined through the streaming controller.

The streaming controller 10 may include a signal processing module 110 for signal processing the obtained IP camera image and a streaming control module 120 for determining a streaming method by analyzing the signal-processed image data. have.

The signal processing module 110 may be configured as a DSP or a codec, and plays a role in image processing in a digital format for wireless transmission of analog image data of an IP camera.

The streaming control module 120 determines streaming of one of full streaming (full streaming) when streaming for all frames is in an active state and partial streaming (partial streaming) in an idle state that requires streaming for some frames. play a role

More specifically, the streaming control module 120 determines whether the operation is in an active state or an idle state by comparing it with a reference change value set by the user when a change value of a frame is input from the signal processing module 110 When the state determined according to the module 121 and the state determination module is an active state, the frame is input to the active processing module, and in the idle state, the frame separation module 122 and the frame separation module input the frame to the idle processing module It can be configured to include an active state streaming module 123 for transmitting the active frame input by the module in full streaming and an idle state streaming module 124 for transmitting the idle frame input by the frame separating module in partial streaming. have.

The state determination module 121 compares the change value of the frame with the reference change value set by the user, and when the change value of the frame is equal to or greater than the reference change value, determines the active state by judging by motion detection, and determines the change value of the frame If it is less than this reference change value, it is considered that there is no motion detection and it is determined as an idle state.

Here, the reference change value can be arbitrarily set by the user, and the entirety of which a change in the motion of the subject being photographed is sensed through the IP camera 30 can be set as the reference change value, and the change in motion in a specific location or area can be set as the reference change value. Only when it is detected, it can be set as the reference change value.

For example, when the IP camera captures an image including an access door, only when a change in the subject in the access door area is detected may be set as the reference change value.

The frame separation module 122 is responsible for selecting and inputting a frame between the active state streaming module 123 and the idle state streaming module 124 according to the state determined by the state determination module 121. .

The active state streaming module 123 controls the input frame to be transmitted through full streaming.

And the idle state streaming module 124 controls the input frame to be transmitted through partial streaming.

The idle state streaming module 124 receives the idle frames and transmits them to the wireless network at a transmission period until the corresponding frames fill the set number of frames per second. It enters a sleep period for as long as it and controls so that idle frames are not transmitted.

In addition, when the idle state streaming module 124 exceeds the time of the sleep period, the process of receiving the frame again and transmitting it to the network is switched from the sleep period to the transmission period, and the first frame to be transmitted to the network is a key frame. to be.

3 is a reference diagram for setting a transmission period in an idle state according to an embodiment of the present invention.

3, when the transmission period setting in the idle state according to an embodiment of the present invention is changed from the active state to the idle state due to motion detection, the stream input to the active state streaming module 123 is idle state streaming It is inputted to the module 124 .

At this time, if it is not a key frame, the active state streaming module 123 continues to participate in stream transmission. When the state is changed to the idle state and a key frame is input for the first time, the active state streaming module 123 stops processing, and the key frame is input to the idle frame processing module.

When a frame is input to the idle state streaming module 123, it is discriminated whether it is the first keyframe or not, and if it is the first keyframe, the number of frames is counted when the starting point for counting starts.

Afterwards, it is determined whether the number of frames has been processed as much as the number of frames previously determined by the user, and if the condition is satisfied, the sleep cycle is entered as much as the sleep interval predetermined by the user.

The sleep period is woken up when the interval time has expired or when motion detection occurs. When the sleep interval time has expired, the next idle frame is processed. When motion is detected, the state becomes an active state, and after the frame wakes up the active state streaming module 123 again, the idle state streaming module 124 transitions to a sleep state.

4 shows that the frame buffer according to a preferred embodiment of the present invention is further included.

Referring to FIG. 4 , the streaming control module 120 may further include a frame buffer 125 for temporarily storing a frame when the state is changed.

When the state determination module 121 is switched from the idle state to the active state due to motion detection, or the active state is switched from the active state to the idle state due to the release of the motion detection, the signal processing module 121 does not immediately change the state for a set time. Stores the frame output from the frame buffer 125, and when the switched state is maintained for a set time, the frame stored in the frame buffer 125 is converted into the active state streaming module 123 or idle state streaming corresponding to the switched state. input to module 124 .

If the state is changed again within the set time, the previous state is maintained, and temporary data stored in the frame buffer 125 is reset.

When the idle state streaming module detects motion in the idle state and immediately switches to the active state, it is switched to the active state even if full streaming is unnecessary due to an error in motion detection or a minute change, and the frame is input through the active state streaming module and fully streamed. Unnecessary traffic may be consumed.

Therefore, when a motion is detected and the state is switched from an idle state to an active state, frame data is temporarily stored in the frame buffer until the motion detection is maintained for a set time, and when the motion detection is maintained for a set time, the frame stored in the frame buffer is stored. It is input to the active state streaming module 123 and is controlled to be transmitted to full streaming.

And when the motion detection is released within a set time and the active state is switched back to the idle state, the frame stored in the frame buffer is maintained in the previous state and input to the idle state streaming module 124 to be transmitted as partial streaming.

On the other hand, the wireless image transmitter 20 according to the present invention is connected to the IP camera 30, receives image data from the IP camera, and a port module 210 for inputting the received image data to the signal processing module 121; It may be configured to include a setting control module 220 for controlling the settings of the IP camera and a network module 230 for transmitting through a wireless network according to a streaming method determined from the streaming control module 120 .

The port module 210 may be configured with RS485, which is a kind of serial communication protocol standard supporting a home network.

And the setting control module 220 serves to control the settings necessary for the operation of the IP camera 30 , and in particular, an operation of distinguishing an active state and an idle state for the image data obtained from the IP camera 30 . It is responsible for controlling the setting of the reference change value for detection.

Here, the setting control module 220 may set a reference change value, which is a reference for motion detection, differently for each time period.

For example, when there is a large change in the subject during the day, the reference change value can be set to a small range, and when there is a large change in the subject at night, the reference change value can be controlled by setting a wide range.

In addition, the setting control module 220 serves to provide an interface for selecting one of the active mode, the idle mode, and the mixed mode.

More specifically, the active mode means a mode in which full streaming is always performed in an active state regardless of motion detection, and the idle mode means a mode in which partial streaming is always performed in an idle state regardless of motion detection, The mixed mode refers to a mode for controlling streaming by switching between an active state and an idle state according to motion detection.

And the setting control module 220 can be configured to set the mode differently for each time zone.

For example, if the IP camera is a surveillance camera, and during the day when there is a lot of change in the subject, it operates in the active mode, and at night, when the change in the subject is small, it is set to operate in the mixed mode, reflecting the characteristics of the IP camera by time of day. can be set differently.

The network module 230 may include a TCP/IP 231 which is a set of communication protocols used for data transmission and a communication module 232 for high-speed data transmission.

The communication module 232 may be configured as a 4G mobile communication technology, LTE (Long Term Evolution) module, or 5G mobile communication technology, IMT-2020 (International Mobile Telecommunications-2020) module.

The present invention enables real-time streaming of video data through a wireless network even when the IP camera 30 is installed in a remote or remote location.

5 is a logic flowchart for an idle state and an active state and frame streaming according to an embodiment of the present invention.

Referring to FIG. 5 , when image data photographed from the IP camera is received by the wireless image transmitter/receiver 20, the digital signal is processed through the signal processing module 110, and the digital signal-processed frame is analyzed by the state determination module 121. It is determined whether this state is active or idle.

That is, the state determination module 121 compares the displacement value of the existing frame and the new frame, and then compares the displacement value with the reference displacement value for motion detection set by the user. It is divided into cases, a case that does not exceed the threshold value is determined as an idle state, and a case that does not exceed the threshold value is determined as an active state.

When it is in the active state, the compressed stream to be transmitted goes into the active state streaming module that processes the active state, and a full streaming operation of transmitting all streams is performed.

And when it enters the idle state, the compressed stream to be transmitted goes into the idle state streaming module that handles the idle state, and, referring to the threshold value set by the user, transmits only some of the stream frames generated in the idle state, and then sleeps It enters the sleep phase by an interval (sleep interval).

In the setting for the operation in the idle state, there is no motion detection of the entire area to be monitored or no motion detection of the specific area to be monitored during idle. According to the method and value of the threshold, the frame is transmitted at a certain level of interval, and the sleep phase is entered.

For reference, the frame transmission method and period in the idle state may be as follows.

- Idle frame rate when transmitting: When transmitting frames in idle state, the number of frames per second to transmit before going to sleep mode

- Idle mode interval rate after transmission: The interval of a short break after continuously transmitting frames in the idle state

In the active state operation, the state judgment module compares the displacement values of the existing frame and the new frame, and then, in the case of an active state by the displacement value, the entire frame input from the signal processing module is continuously processed according to the set value. send to

In the operation in the idle state, the state determination module compares the displacement values of the existing frame and the new frame, and when the displacement value is in the idle state, the period set by the user among all frames input from the signal processing module is Transmit a specific frame according to the method.

In the case of the active state, when the streaming service starts, the mode is switched to active and a full frame is transmitted. It is transmitted with reference to a value such as a key frame repetition rate. Frame transmission first finds a keyframe, and if there is no keyframe, requests a keyframe from the signal processing module, and then starts transmission.

On the other hand, with respect to the active duration, after starting frame transmission, it continues until the time threshold of the active duration set by the user is exceeded. .

In the case of the idle state, the first keyframe is found, and if there is no keyframe, it requests the keyframe from the signal processing module and starts transmitting from that point. At this time, the number of frames per second is transmitted at the frame rate in the idle state of the threshold set by the user.

Regarding the frame count, when performing transmission after sleep in the idle state, the number of frames to be transmitted per second depends on the threshold value set by the user. Afterwards, when the motion detection (Intelligent detect) does not occur, the frame transmission is stopped in idle and changes to a sleep mode, and the sleep time follows the threshold value set by the user.

Regarding sleep, when the module block wakes up after completing transmission in the idle state and entering the sleep mode, it depends on the sleep timeout and motion detection requirements. The sleep timeout requirement is that when the sleep time is exceeded, the processing block wakes up, and at this time, it repeats the operation of retransmitting the frame count.

At this time, transmission starts from the keyframe. The motion detection requirement allows a block to wake up, change its state to an active state, and go through processing in the active state when a motion detection is detected.

On the other hand, an interrupt in the idle state is in the middle of transmission. During sleep or in any state, when motion is detected, it immediately changes to an active state and undergoes processing in the active state again.

In the detailed description of the present invention described above, it has been described with reference to the preferred embodiment of the present invention, but the protection scope of the present invention is not limited to the above embodiment, and those of ordinary skill in the art It will be understood that various modifications and variations of the present invention can be made without departing from the spirit and scope of the present invention.

10: streaming controller 20: wireless video transmitter and receiver
30: IP camera 40: administrator terminal

Claims (6)

a streaming controller for controlling streaming by determining one of full streaming and partial streaming through intelligent analysis according to motion detection of the acquired IP camera image;
It is connected to the IP camera and transmits the video data of the IP camera to the streaming controller, controls the settings of the IP camera, and transmits image frame data through a wireless network according to the streaming method determined through the streaming controller. Intelligent video streaming transmission control device comprising a video transceiver.
The method of claim 1,
The streaming controller
a signal processing module for signal processing the acquired IP camera image;
a streaming control module for determining a streaming method by analyzing the signal-processed image data;
The streaming control module
a state determination module for determining whether an operation is in an active state or an idle state by comparing a reference change value corresponding to a motion detection set by a user when a frame change value is input from the signal processing module;
a frame separation module for inputting a frame to the active processing module when the state determined by the state determination module is an active state, and for inputting a frame to the idle processing module when in an idle state;
an active state streaming module for transmitting the active frame input by the frame separating module in full streaming;
Intelligent video streaming transmission control device, characterized in that it comprises an idle state streaming module for transmitting the idle frame input by the frame separation module in partial streaming.
3. The method of claim 2,
The streaming control module
a frame buffer for temporarily storing a frame when a state is changed;
When the state determination module is switched from the idle state to the active state due to motion detection, or the active state is switched from the active state to the idle state due to the release of the motion detection, the frame output from the signal processing module is not changed immediately for a set time. Intelligent video streaming transmission control device, characterized in that the frame is stored in the frame buffer, and when the switched state is maintained for a set time, the frame stored in the frame buffer is input to the active state streaming module or idle state streaming module corresponding to the switched state. .
3. The method of claim 2,
The wireless image transmitter and receiver
a port module connected to the IP camera to receive image data from the IP camera and input the received image data to the signal processing module;
a setting control module for controlling the setting of the IP camera;
Intelligent video streaming transmission control device comprising a network module for transmitting through a wireless network according to the streaming method determined from the streaming control module.
5. The method of claim 4,
The setting control module is
controlling settings necessary for the operation of the IP camera, and controlling the setting of a reference change value for motion detection that distinguishes an active state and an idle state for image data acquired from the IP camera;
Active mode that controls to always stream in an active state regardless of motion detection, idle mode that controls partial streaming in an idle state regardless of motion detection, and controls streaming by switching between active and idle states according to motion detection Intelligent video streaming transmission control device, characterized in that it provides an interface for selecting one of the mixed modes.
6. The method of claim 5,
The setting control module is
Intelligent video streaming transmission control device, characterized in that it provides an interface that can set the operation mode differently for each time zone.

Images