CN115865879A - Unattended video remote monitoring system - Google Patents
Unattended video remote monitoring system Download PDFInfo
- Publication number
- CN115865879A CN115865879A CN202211514371.2A CN202211514371A CN115865879A CN 115865879 A CN115865879 A CN 115865879A CN 202211514371 A CN202211514371 A CN 202211514371A CN 115865879 A CN115865879 A CN 115865879A
- Authority
- CN
- China
- Prior art keywords
- video
- control center
- remote
- data
- remote end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention belongs to the field of remote video monitoring and discloses an unattended video remote monitoring system. The system comprises remote end monitoring equipment, video source equipment, remote end video service, a streaming media server, a communication link and a control center; the system comprises a method for remotely acquiring a video source through a network in an unattended state, a method for automatically reconnecting when the network is abnormal, a method for monitoring the system state and a method for efficiently utilizing the network bandwidth based on image characteristic recognition. The system design can ensure that the remote video is transmitted to the control center in a low delay way under the normal working state, and can feed back the reason of the abnormality in time under the abnormal stateThe method can save network bandwidth, is beneficial to realizing bandwidth self-adaption of remote video transmission, and further realizes more reasonable bandwidth allocation and better overall utility 。
Description
Technical Field
The invention relates to the field of remote video monitoring, and particularly discloses a design scheme of a remote video monitoring system with remote video services (pulling, pushing video streaming and target detection) and a control center (video display and SEI (solid image interface) extended data analysis and feedback), and a solution of a streaming data transmission problem under the condition of unstable actual communication link.
Background
The remote video monitoring system is a program for monitoring a remote scene in real time through a communication link. In recent years, with the rapid development of computer hardware and video image processing technology, the data scale is gradually enlarged, especially with the increase of monitoring nodes, the data volume needing to be transmitted is more and more increased, however, in the actual long-distance transmission, because the communication guarantee condition is not good, how to perform the streaming data transmission with high quality for a long time, realize quick response, ensure the effectiveness of real-time monitoring and the confidentiality of sensitive information transmission, and prevent the loss of important events in monitoring becomes an important research subject of the current special monitoring system.
The traditional technical scheme generally focuses on coding compression of video images, for example, a local coding mode is adopted by using an H264 format, so that the communication bandwidth required by high-definition image transmission can be reduced to a certain extent, but a reasonable stream pushing mechanism is designed to further save the communication bandwidth by considering the sparsity of the occurrence time of a target in each video line.
Disclosure of Invention
The invention provides an unattended video remote monitoring system, which aims to adapt to a real network environment and save communication link bandwidth to the maximum extent, and realize that video stream data of a server can be transmitted to a local terminal to be displayed at any time and place.
The technical scheme adopted by the invention is as follows:
an unattended video remote monitoring system comprises remote end monitoring equipment, video source equipment, remote end video service, a streaming media server, a communication link and a control center;
the remote-end video service is used for acquiring video source equipment data in real time by using an audio and video decoding tool, identifying images in real time by using a yolo deep learning algorithm on each frame of data of the video source data, if an important target is detected, sequencing the priority of a plurality of paths of video streams according to the identification result and the importance degree, inserting SEI (solid electrolyte interphase) extension data into the video streams and pushing the video streams to a streaming media server, and otherwise, pushing the video streams to the streaming media server according to the acquired video stream sequence; the video processing system is also used for reordering the importance degree of the multi-channel video stream according to the result fed back by the control center;
the streaming media server is used for transmitting the video stream to the control center through a communication link in a streaming protocol mode;
the control center is used for receiving the video stream from the streaming media server, analyzing SEI extended data of the video stream, receiving the video stream data if the extracted SEI extended data is consistent with a judgment result of the control center, and sending the judgment result of the control center to the remote end video service if the extracted SEI extended data is inconsistent with the judgment result of the control center;
the remote end monitoring equipment is used for establishing state monitoring with the remote end video service, the communication link and the video source equipment and transmitting a state monitoring result to the control center.
Further, the remote end video service is also used for caching other video streams except the video stream with the highest priority, pushing the cached video information to the control center through the streaming media server and the communication link, and pushing the video stream according to the on-demand instruction and the current conversion instruction of the control center; the control center is also used for generating a demand list from the cached video information and sending a demand instruction to the remote end video service according to the demand condition or sending a commutation instruction to the remote end video service.
Furthermore, the remote end monitoring device is further configured to transmit the real-time bandwidth information to a remote end video service, and when the bandwidth is smaller than a threshold, the remote end video service selects a preset frame rate gear to push a video stream while keeping the resolution unchanged.
Compared with the background technology, the invention has the advantages that:
(1) The invention has a feedback mechanism and is more intelligent. The system has a certain target recognition function and an image processing function, can automatically push a video circuit with a recognized target to a client (a control center), and the control center can also feed back a self judgment result to a remote end video service to correct errors of a target feature extraction and detection module. The user can quickly master effective video monitoring information, managers of security or other projects can be effectively helped to improve monitoring efficiency, and service quality is improved.
(2) Has higher stability. The system can automatically reconnect under the condition of network abnormity without restarting application service until the network condition is good, and ensures the transmission of video stream data.
(3) It is more expandable. The system has strong expandability, can realize the monitoring of multiple devices at large visual angles, and can easily realize that multiple computers pull the same video source data for retrieval because the monitoring range is expanded mainly by increasing the monitoring points of a server (remote video service) and a client (control center) only recognizes and pulls the source data by the ip address.
(4) It has more flexibility. The system can automatically push the video with the most monitoring value to the control center, and the user can freely call and read the videos in other lines or a cache region.
Drawings
FIG. 1 is a schematic block diagram of a remote video surveillance system of the present invention.
Fig. 2 is a flow chart of the decision for switching video streams of the remote video surveillance system according to the present invention.
Detailed Description
The invention is further explained below with reference to the drawings.
When the remote communication link transmits data, the remote communication link is limited by small bandwidth of the remote communication link and is easy to be interfered, and the situation that the actual data occupies bandwidth far larger than the actual communication bandwidth is easy to occur, so that the requirement of real-time monitoring effectiveness under the background is ensured. In order to ensure the flexibility of the system, the control center can call and read videos of other lines through the video line switching instruction, and in addition, the invention also designs a cache on-demand mechanism, which can put the videos in the lines with lower priority into a cache area and report the cache video information to the control center to generate an on-demand list to remind an operator of on-demand viewing. In addition, the server can receive real-time bandwidth information from the remote monitoring equipment, if the real-time bandwidth is too small, the preset frame rate gear is selected to push the video stream under the condition that the resolution ratio is not changed, and the technical scheme can solve the problem that the video stream data is transmitted with high quality for a long time under the condition that a communication link is limited.
Because real-time monitoring is realized by using transmitted video stream data, and a communication link is easily influenced by factors such as obstacles, weather and strong signal interference sources, and the like, the stream data transmission of a remote video monitoring system is further influenced, and the system needs to establish a disconnection reconnection mechanism to ensure the stable operation of the system.
Meanwhile, the service end (remote end video service) is in an unattended state for a long time, the service end (remote end video service) needs to assist the equipment monitoring system to send the current working state information of all the equipment of the remote end to the client end (control center), and the equipment monitoring system needs to maintain a normal working state under the condition of considering emergency (such as power failure, service end power failure or communication link failure).
As shown in fig. 1, an unattended video remote monitoring system includes a remote end monitoring device, a video source device, a remote end video service, a streaming media server, a communication link, and a control center;
the remote-end video service is used for acquiring video source equipment data in real time by using an audio and video decoding tool, recognizing images in real time by using a yolo deep learning algorithm on each frame of data of the video source data, if an important target is detected, sequencing a plurality of paths of video streams according to the recognition result and the importance degree, inserting the video streams into SEI (solid electrolyte interphase) extension data and pushing the SEI extension data to a streaming media server, otherwise, pushing the SEI extension data to the streaming media server according to the sequence of the acquired video streams, and selecting a preset frame rate gear to push the video streams under the condition of keeping the resolution unchanged when the bandwidth is less than a threshold value; the video processing system is also used for reordering the importance degree of the multi-channel video stream according to the result fed back by the control center; the video streaming server is also used for caching other video streams except the video stream with the highest priority, pushing the cached video information to the streaming media server, and pushing the video stream according to the on-demand instruction and the current conversion instruction of the control center;
the streaming media server is used for transmitting the video stream and the cached video information to the control center through a communication link in a streaming protocol mode;
the control center is used for receiving the video stream from the streaming media server, analyzing SEI extended data of the video stream, receiving the video stream data if the extracted SEI extended data is consistent with a judgment result of the control center, and sending the judgment result of the control center to the remote end video service if the extracted SEI extended data is inconsistent with the judgment result of the control center; the control center is also used for generating a demand list from the cached video information and sending a demand instruction to the remote end video service according to the demand condition or sending a commutation instruction to the remote end video service.
The remote end monitoring equipment is used for establishing state monitoring with the remote end video service, the communication link and the video source equipment and transmitting a state monitoring result to the control center. And also for transmitting real-time bandwidth information to a remote-end video service.
The implementation flow is shown in fig. 2:
step 1: and (5) building a streaming media server and opening remote end monitoring equipment. The streaming media server has the main functions of transmitting videos to the client in a streaming protocol mode for a user to pull streaming data, is positioned as an operation-level Internet live broadcast server cluster, has the characteristic of easy deployment, provides various installation schemes, monitoring indexes, observable logs and the like, can be adapted to different operating systems, can realize uniform deployment, can set a starting self-starting option after successful deployment, and realizes the functions of collecting, caching, scheduling and transmitting and playing streaming media contents. After the remote monitoring device is opened and the connection between the remote monitoring device and the server-side device is established, the state information from the monitoring device and the bandwidth size of the current link can be received in the control center.
Step 2: and the server (remote end video service) pulls the video source data, realizes the conversion and encapsulation of the stream data and pushes the video stream to the stream media server. Because the original H264 code stream pulled from the video source meets the rtsp real-time stream protocol, the protocol is mainly used for controlling data transmission with real-time characteristics, the protocol does not transmit data, most business requirements are to convert the rtsp video stream into a video (in formats such as MP4, flv, m3u8, rtmp and the like) capable of being played on a mainstream playing channel such as a mobile phone or a browser, most internet live broadcast cloud services and streaming media servers in the market support rtmp push stream input, and what we need to do is to transcode the rtsp stream into the rtmp stream and push the rtmp stream to a live broadcast address of a public cloud or streaming media server. The rtmp stream adopts FLV as a packaging format, so that format parameters of an output stream media need to be specified as a correct packaging format, and stream data sent to the stream media needs to be subjected to delay processing, which is caused by a fast speed of ffmpeg processing data, and all data can be sent out at an instant, so that a stream media server cannot bear the delay processing, so that an av _ use function built in ffmpeg needs to be used in a video transmission part for sleep processing, so as to realize delay sending between video frames, but bare stream data without packaging (such as H264 bare stream) does not contain video frame parameters (such as pts, dts and duration), the video transmission part needs to calculate relevant parameters by itself when sending the stream data, and corresponding delay caused when decoding and displaying can be eliminated only when a buffer space is set in an av _ di structure. Considering that remote network communication is susceptible to various factors, no matter the service end or the control center starts the pull thread, a timer needs to be started to monitor whether the pull thread is running or closed, and if the pull thread is closed, the pull thread needs to be restarted.
And step 3: and after high-definition image acquisition and target detection are carried out, one path of video is pushed to a control center according to the priority order. Generally, the optoelectronic device has an SDK or other corresponding external interface available for developers to capture high definition images, and for example, a gram _ image _ async function is provided in a HALCON standard machine vision algorithm package to make the image acquisition and processing parallel. In the invention, an original picture and a background picture are selected for difference, gaussian filtering and denoising are carried out, meanwhile, the threshold is selected to change the image into a binary image to connect pixels and mark the image to realize the operation of image segmentation, and for the sake of simplicity, an opencv dnn module is used to call a YOLO v4 training model to complete the prediction of a target result. The identified characteristic result is inserted into a video frame through SEI extended data to ensure that the video data and the identification result are accurately synchronized, priority sequencing is carried out on multi-channel video stream data according to the importance degree, the video data with the highest priority is pushed to a stream pushing address of a preset stream media server to be pulled by a control center, other line data are not pushed temporarily, if targets also appear in other line data, a server side sends prompt information to the control center and generates an on-demand list, and if the importance degree is the same, a line with the higher confidence coefficient of the target result is selected as the higher priority. The SEI enhancement information has excellent compatibility, and if other third-party player software is used for pulling video stream data pushed to the stream pushing address and cannot analyze SEI, the player can automatically ignore SEI data packets and does not influence the normal playing of the video. When the current link bandwidth size information sent by the remote end monitoring equipment is received and changed, the server end can automatically log in the background of the photoelectric equipment and select the matched video frame rate according to the current bandwidth size and the preset frame rate gear.
And 4, step 4: the control center decodes the display stream data and performs feedback regulation. After the control center pulls rtmp stream data from the streaming media server, the rtmp stream data is decoded and played by using ffmpeg, in addition, the control center can compare the analyzed data with the self judgment result after analyzing the SEI data, if the analyzed data is different from the SEI data, a feedback instruction is sent to the server side, the server side can carry out priority rearrangement on all the video stream data according to the feedback result, the video with the highest importance degree is pushed to the control center again, if the control center selects the video in the on-demand list, the control center can send an instruction to the server side, the server side can immediately pause the pushing flow, reset an encoder and push the selected local video to the streaming media server in a cache region according to the instruction, the streaming continues according to the priority sequence after the on-demand is finished, if the control center selects other line videos, the control center can send corresponding current conversion instructions to the server side, and the server side can push the stream again according to the appointed switching line.
Claims (3)
1. An unattended video remote monitoring system is characterized by comprising remote end monitoring equipment, video source equipment, remote end video service, a streaming media server, a communication link and a control center;
the remote-end video service is used for acquiring video source equipment data in real time by using an audio and video decoding tool, identifying images in real time by using a yolo deep learning algorithm on each frame of data of the video source data, if an important target is detected, sequencing the priority of a plurality of paths of video streams according to the identification result and the importance degree, inserting SEI (solid electrolyte interphase) extension data into the video streams and pushing the video streams to a streaming media server, and otherwise, pushing the video streams to the streaming media server according to the acquired video stream sequence; the video processing system is also used for reordering the importance degree of the multi-path video stream according to the result fed back by the control center;
the streaming media server is used for transmitting the video stream to the control center through a communication link in a streaming protocol mode;
the control center is used for receiving the video stream from the streaming media server, analyzing SEI extended data of the video stream, receiving the video stream data if the extracted SEI extended data is consistent with a judgment result of the control center, and sending the judgment result of the control center to the remote end video service if the extracted SEI extended data is inconsistent with the judgment result of the control center;
the remote end monitoring equipment is used for establishing state monitoring with the remote end video service, the communication link and the video source equipment and transmitting a state monitoring result to the control center.
2. The unattended video remote monitoring system according to claim 1, wherein the remote video service is further configured to cache video streams other than the video stream with the highest priority, and push the cached video information to the control center through the streaming media server and the communication link, and is further configured to push the video stream according to a request command and a commutation command of the control center; the control center is also used for generating a demand list from the cached video information and sending a demand instruction to the remote end video service according to the demand condition or sending a commutation instruction to the remote end video service.
3. The unattended video remote monitoring system according to claim 1, wherein the remote end monitoring device is further configured to transmit real-time bandwidth information to the remote end video service, and the remote end video service selects a preset frame rate gear to push the video stream while keeping a resolution unchanged when the bandwidth is less than a threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211514371.2A CN115865879A (en) | 2022-11-30 | 2022-11-30 | Unattended video remote monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211514371.2A CN115865879A (en) | 2022-11-30 | 2022-11-30 | Unattended video remote monitoring system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115865879A true CN115865879A (en) | 2023-03-28 |
Family
ID=85667948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211514371.2A Pending CN115865879A (en) | 2022-11-30 | 2022-11-30 | Unattended video remote monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115865879A (en) |
-
2022
- 2022-11-30 CN CN202211514371.2A patent/CN115865879A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10728594B2 (en) | Method and apparatus for transmitting data of mobile terminal | |
| US10250664B2 (en) | Placeshifting live encoded video faster than real time | |
| CN110662114B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN110661752A (en) | Plug-in-free real-time video playing system and method | |
| CN103647954A (en) | Mobile video monitoring system based on 3G channel and intelligent terminal | |
| CN104918137A (en) | Method enabling spliced screen system to play videos | |
| CN110933470B (en) | Video data sharing method | |
| WO2024022317A1 (en) | Video stream processing method and apparatus, storage medium, and electronic device | |
| KR20150106351A (en) | Method and system for playback of motion video | |
| CN111866526B (en) | Live broadcast service processing method and device | |
| CN110958431A (en) | Multi-channel video compression post-transmission system and method | |
| CN113225585A (en) | Video definition switching method and device, electronic equipment and storage medium | |
| CN101800886A (en) | Video pushing method based on home network | |
| CN116527852A (en) | Real-time video monitoring system | |
| CN105451004A (en) | Intelligent set top box, and media processing system and method supporting video communication monitoring | |
| CN111093082A (en) | Method and system for live broadcasting of streaming media | |
| CN113556612A (en) | Method and system for playing H.265 video stream on browser | |
| CN113747191A (en) | Video live broadcast method, system, equipment and storage medium based on unmanned aerial vehicle | |
| CN113596112A (en) | Transmission method for video monitoring | |
| CN104639979A (en) | Video sharing method and system | |
| CN114339146B (en) | Audio and video monitoring method and device, electronic equipment and computer readable storage medium | |
| CN115865879A (en) | Unattended video remote monitoring system | |
| CN116112620A (en) | Processing method and system for improving video stream multipath merging stability | |
| CN102164272A (en) | Method for reducing cloud deck control delay in network video monitoring platform | |
| CN110798700B (en) | Video processing method, video processing device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |