CN110087040B

CN110087040B - Monitoring video calling method and system

Info

Publication number: CN110087040B
Application number: CN201910364254.4A
Authority: CN
Inventors: 白羽; 沈军; 王军; 郭忠平
Original assignee: Visionvera Information Technology Co Ltd
Current assignee: Visionvera Information Technology Co Ltd
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2020-11-03
Anticipated expiration: 2039-04-30
Also published as: CN110087040A

Abstract

The embodiment of the application provides a monitoring video calling method, which is applied to a video network, wherein the video network comprises an Mserver, a shared platform server and a monitoring access server, the monitoring access server is also in communication connection with a plurality of monitoring devices in the Internet, the monitoring access server receives a first video calling signaling sent by an IE plug-in through the Mserver and the shared platform server, requests a first video stream from a target monitoring device according to the first video calling signaling, sends the first video stream to the shared platform server if the first video stream is received, and generates a first error code if the first video stream is not received; if the first video stream is not successfully sent by the sharing platform server, a second error code is generated, and if the first video stream is received by the IE plug-in, the IE plug-in plays the first video stream; and if the first error code or the second error code is received, displaying the first error code or the second error code. The method and the device realize that the stream adjusting result can be displayed on the IE plug-in no matter whether the monitoring video is successfully called or not.

Description

Monitoring video calling method and system

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a surveillance video retrieval method and a surveillance video retrieval system.

Background

The video network is a real-time switching platform, is a higher-level form of the internet, faces to potential huge video flow on the network, adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the Ethernet defect on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, directly bears an IP data packet, does not need any format conversion in the whole network range of user data, and can realize the real-time transmission of the whole network high-definition video which can not be realized by the internet at present.

Therefore, the internet of view has pushed many internet applications to high definition video, unification, high definition face-to-face. Finally, world no distance is realized, and the distance between people in the world is only the distance of one screen; on the other hand, the video network has the flexibility, simplicity and low price of packet switching, simultaneously has the quality and safety guarantee of circuit switching, and realizes the seamless connection of the whole network switching type virtual circuit and the data format for the first time in the communication history.

At present, when a user calls a monitoring video through an internet of things via a video network through an IE (Internet of things) plug-in the internet, the user tries to repeat operations under the condition that the video is not displayed at a later time, and system crash is often caused; and the user can not know the reason of the error even if the video is not displayed at a later time, and can solve the problem only after spending a large amount of time to troubleshoot the problem, which obviously seriously affects the operating efficiency of the video networking service.

Disclosure of Invention

In view of the above problems, embodiments of the present application are proposed to provide a surveillance video retrieval method and a surveillance video retrieval system that overcome or at least partially solve the above problems.

In order to solve the above problems, an embodiment of the present application discloses a monitoring video invoking method, which is applied to a video network, where the video network includes a monitoring network management scheduling platform server Mserver, a shared platform server communicatively connected to the Mserver, and a monitoring access server communicatively connected to the shared platform server, the monitoring access server is further communicatively connected to a plurality of monitoring devices in the internet, and the Mserver is further communicatively connected to a browser IE plug-in the internet; the method comprises the following steps:

the IE plug-in generates a first video calling signaling which accords with an internet protocol according to the received stream adjusting operation, and sends the first video calling signaling to the Mserver;

the Mserver converts the first video calling signaling into a second video calling signaling which accords with a video networking protocol, and sends the second video calling signaling to the shared platform server;

the monitoring access server receives the second video calling signaling forwarded by the sharing platform server, requests a target monitoring device for a first video stream according to the second video calling signaling, and detects whether the first video stream is received; if the first video stream is received, sending the first video stream to the sharing platform server; if the first video stream is not received, generating a first error code according to the current offline state of the target monitoring equipment or the current video networking resource state of the monitoring access server, and sending the first error code to the shared platform server;

when receiving the first error code, the shared platform server sends the first error code to the Mserver; or when the first video stream is received, converting the first video stream into a second video stream conforming to an internet protocol, and when the second video stream is sent to the IE plugin, detecting the state of sending the second video stream; if the state is a sending failure state, generating a second error code, and sending the second error code to the Mserver;

the Mserver converts the first error code into a first text error message which conforms to an initial session sip protocol, or converts the second error code into a second text error message which conforms to the sip protocol; sending the first text error message or the second text error message to the IE plug-in;

if the IE plug-in receives the second video stream, playing the second video stream; and if the first text error message or the second text error message is received, displaying the first text error message or the second text error message.

In order to solve the above problem, an embodiment of the present application further discloses a monitoring video invoking system, where the system is applied to a video network, the video network includes a monitoring network management scheduling platform server Mserver, a shared platform server in communication connection with the Mserver, and a monitoring access server in communication connection with the shared platform server, the monitoring access server is in communication connection with a plurality of monitoring devices in the internet, and the Mserver is also in communication connection with a browser IE plug-in the internet; the system comprises:

the video stream signaling module is positioned in the IE plug-in and used for generating a first video calling signaling which accords with an Internet protocol according to the received stream adjusting operation and sending the first video calling signaling to the Mserver;

the signaling conversion module is positioned in the Mserver and used for converting the first video calling signaling into a second video calling signaling which accords with a video networking protocol and sending the second video calling signaling to the shared platform server;

a video stream detection module, located in the monitoring access server, and configured to receive the second video call signaling forwarded by the sharing platform server, request a first video stream from a target monitoring device according to the second video call signaling, and detect whether the first video stream is received;

the video stream forwarding module is positioned in the monitoring access server and used for receiving the first video stream and sending the first video stream to the sharing platform server;

a first error code generation module, located in the monitoring access server, configured to generate a first error code according to a current offline state of the target monitoring device or a current video networking resource state of the monitoring access server if the first video stream is not received, and send the first error code to the shared platform server;

an error code forwarding module, located in the shared platform server, and configured to send the first error code to the Mserver when the first error code sent by the monitoring access server is received;

a video sending state detection module, located in the shared platform server, for converting the first video stream into a second video stream conforming to an internet protocol when receiving the first video stream sent by the monitoring access server, and detecting a state of sending the second video stream when sending the second video stream to the IE plugin;

the second error code generation module is positioned on the shared platform server and used for generating a second error code and sending the second error code to the Mserver if the state is a sending failure state;

an error code conversion module, located in the Mserver, for converting the first error code into a first text error message conforming to a session initiation sip protocol, or converting the second error code into a second text error message conforming to a sip protocol; sending the first text error message or the second text error message to the IE plug-in;

the video playing module is positioned in the IE plug-in and is used for playing the second video stream if the second video stream successfully sent by the sharing platform server is received;

and the display module is positioned in the IE plug-in and is used for displaying the first text error message or the second text error message if the first text error message or the second text error message sent by the Mserver is received.

The embodiment of the application has the following advantages:

firstly, in the embodiment of the application, when an IE plugin sends a first video calling signaling, the Mserver converts the first video calling signaling into a second video calling signaling conforming to a video networking protocol, and when receiving the second video calling signaling, a monitoring access server can detect whether a first video stream of a target monitoring device is received; if so, sending the first video stream to a sharing platform server; and if the first error code is not received, generating a first error code, converting the first video stream into a second video stream conforming to the Internet protocol by the sharing platform server, detecting the sending state of the second video stream when the second video stream is sent to the IE plug-in, and if the second video stream is failed to be sent, generating a second error code. And when receiving the first error code or the second error code, the Mserver converts the first error code or the second error code into corresponding first text error information and second text error information and sends the first text error information and the second text error information to the IE plug-in. When the IE plug-in receives the second video stream, playing the second video stream; and when the first text error message or the second text error message is received, displaying the first text error message or the second text error message. Therefore, when the surveillance video is successfully called, a user can directly watch the video played after decoding, when the surveillance video is unsuccessfully called, the user can also quickly determine the reason of the unsuccessfully called surveillance video according to the displayed text error information, and then the user can conveniently find out the problem according to the displayed text error information, so that the user can clearly know the final result of calling the surveillance video, the operation efficiency of the video networking service is improved, and the user experience is also improved.

Secondly, after the video stream is successfully called, the monitoring access server and the sharing platform server can also continuously detect the state of the called video stream, if the target monitoring device is suddenly disconnected in the midway or fails to send the second video stream, text error information can be displayed on the IE plug-in, so that in the process of playing the second video stream, if the video is suddenly interrupted, a user can also troubleshoot the problem according to the displayed text error information.

Drawings

FIG. 1 is a networking schematic of a video network of the present application;

FIG. 2 is a schematic diagram of a hardware architecture of a node server according to the present application;

fig. 3 is a schematic diagram of a hardware architecture of an access switch of the present application;

fig. 4 is a schematic diagram of a hardware structure of an ethernet protocol conversion gateway according to the present application;

fig. 5 is a flowchart illustrating steps of a surveillance video retrieving method according to a first embodiment of the present application;

fig. 6 is an application environment diagram of a surveillance video retrieval method according to a first embodiment of the present application;

FIG. 7 is a flowchart illustrating an alternative example step in another surveillance video retrieval method according to an embodiment of the present application;

FIG. 8 is a flowchart illustrating an alternative example step in another surveillance video retrieval method according to an embodiment of the present application;

FIG. 9 is a flowchart illustrating an alternative example step in another surveillance video retrieval method according to an embodiment of the present application;

FIG. 10 is a flowchart illustrating an alternative example step in another surveillance video retrieval method according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a surveillance video retrieval system according to a second embodiment of the present application.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.

The video networking is an important milestone for network development, is a real-time network, can realize high-definition video real-time transmission, and pushes a plurality of internet applications to high-definition video, and high-definition faces each other.

The video networking adopts a real-time high-definition video exchange technology, can integrate required services such as dozens of services of video, voice, pictures, characters, communication, data and the like on a system platform on a network platform, such as high-definition video conference, video monitoring, intelligent monitoring analysis, emergency command, digital broadcast television, delayed television, network teaching, live broadcast, VOD on demand, television mail, Personal Video Recorder (PVR), intranet (self-office) channels, intelligent video broadcast control, information distribution and the like, and realizes high-definition quality video broadcast through a television or a computer.

To better understand the embodiments of the present application, the following description refers to the internet of view:

some of the technologies applied in the video networking are as follows:

network Technology (Network Technology)

Network technology innovation in video networking has improved the traditional Ethernet (Ethernet) to face the potentially huge first video traffic on the network. Unlike pure network Packet Switching (Packet Switching) or network Circuit Switching (Circuit Switching), the Packet Switching is adopted by the technology of the video networking to meet the Streaming requirement. The video networking technology has the advantages of flexibility, simplicity and low price of packet switching, and simultaneously has the quality and safety guarantee of circuit switching, thereby realizing the seamless connection of the whole network switching type virtual circuit and the data format.

Switching Technology (Switching Technology)

The video network adopts two advantages of asynchronism and packet switching of the Ethernet, eliminates the defects of the Ethernet on the premise of full compatibility, has end-to-end seamless connection of the whole network, is directly communicated with a user terminal, and directly bears an IP data packet. The user data does not require any format conversion across the entire network. The video networking is a higher-level form of the Ethernet, is a real-time exchange platform, can realize the real-time transmission of the whole-network large-scale high-definition video which cannot be realized by the existing Internet, and pushes a plurality of network video applications to high-definition and unification.

Server Technology (Server Technology)

The server technology on the video networking and unified video platform is different from the traditional server, the streaming media transmission of the video networking and unified video platform is established on the basis of connection orientation, the data processing capacity of the video networking and unified video platform is independent of flow and communication time, and a single network layer can contain signaling and data transmission. For voice and video services, the complexity of video networking and unified video platform streaming media processing is much simpler than that of data processing, and the efficiency is greatly improved by more than one hundred times compared with that of a traditional server.

Storage Technology (Storage Technology)

The super-high speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the media content with super-large capacity and super-large flow, the program information in the server instruction is mapped to the specific hard disk space, the media content is not passed through the server any more, and is directly sent to the user terminal instantly, and the general waiting time of the user is less than 0.2 second. The optimized sector distribution greatly reduces the mechanical motion of the magnetic head track seeking of the hard disk, the resource consumption only accounts for 20% of that of the IP internet of the same grade, but concurrent flow which is 3 times larger than that of the traditional hard disk array is generated, and the comprehensive efficiency is improved by more than 10 times.

Network Security Technology (Network Security Technology)

The structural design of the video network completely eliminates the network security problem troubling the internet structurally by the modes of independent service permission control each time, complete isolation of equipment and user data and the like, generally does not need antivirus programs and firewalls, avoids the attack of hackers and viruses, and provides a structural carefree security network for users.

Service Innovation Technology (Service Innovation Technology)

The unified video platform integrates services and transmission, and is not only automatically connected once whether a single user, a private network user or a network aggregate. The user terminal, the set-top box or the PC are directly connected to the unified video platform to obtain various multimedia video services in various forms. The unified video platform adopts a menu type configuration table mode to replace the traditional complex application programming, can realize complex application by using very few codes, and realizes infinite new service innovation.

Networking of the video network is as follows:

the video network is a centralized control network structure, and the network can be a tree network, a star network, a ring network and the like, but on the basis of the centralized control node, the whole network is controlled by the centralized control node in the network.

As shown in fig. 1, the video network is divided into an access network and a metropolitan network.

The devices of the access network part can be mainly classified into 3 types: node server, access switch, terminal (including various set-top boxes, coding boards, memories, etc.). The node server is connected to an access switch, which may be connected to a plurality of terminals and may be connected to an ethernet network.

The node server is a node which plays a centralized control function in the access network and can control the access switch and the terminal. The node server can be directly connected with the access switch or directly connected with the terminal.

Similarly, devices of the metropolitan network portion may also be classified into 3 types: a metropolitan area server, a node switch and a node server. The metro server is connected to a node switch, which may be connected to a plurality of node servers.

The node server is a node server of the access network part, namely the node server belongs to both the access network part and the metropolitan area network part.

The metropolitan area server is a node which plays a centralized control function in the metropolitan area network and can control a node switch and a node server. The metropolitan area server can be directly connected with the node switch or directly connected with the node server.

Therefore, the whole video network is a network structure with layered centralized control, and the network controlled by the node server and the metropolitan area server can be in various structures such as tree, star and ring.

The access network part can form a unified video platform (the part in the dotted circle), and a plurality of unified video platforms can form a video network; each unified video platform may be interconnected via metropolitan area and wide area video networking.

Video networking device classification

1.1 devices in the video network of the embodiment of the present application can be mainly classified into 3 types: server, exchanger (including Ethernet protocol conversion gateway), terminal (including various set-top boxes, code board, memory, etc.). The video network as a whole can be divided into a metropolitan area network (or national network, global network, etc.) and an access network.

1.2 wherein the devices of the access network part can be mainly classified into 3 types: node server, access exchanger (including Ethernet protocol conversion gateway), terminal (including various set-top boxes, coding board, memory, etc.).

The specific hardware structure of each access network device is as follows:

a node server:

as shown in fig. 2, the system mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;

the network interface module 201, the CPU module 203, and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of looking up the address table 205 on the incoming packet, thereby obtaining the direction information of the packet; and stores the packet in a queue of the corresponding packet buffer 206 based on the packet's steering information; if the queue of the packet buffer 206 is nearly full, it is discarded; the switching engine module 202 polls all packet buffer queues for forwarding if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero. The disk array module 204 mainly implements control over the hard disk, including initialization, read-write, and other operations on the hard disk; the CPU module 203 is mainly responsible for protocol processing with an access switch and a terminal (not shown in the figure), configuring an address table 205 (including a downlink protocol packet address table, an uplink protocol packet address table, and a data packet address table), and configuring the disk array module 204.

The access switch:

as shown in fig. 3, the network interface module mainly includes a network interface module (a downlink network interface module 301 and an uplink network interface module 302), a switching engine module 303 and a CPU module 304;

wherein, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the Destination Address (DA), the Source Address (SA), the packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id) and enters the switching engine module 303, otherwise, discards the stream identifier; the packet (downstream data) coming from the upstream network interface module 302 enters the switching engine module 303; the incoming data packet of the CPU module 304 enters the switching engine module 303; the switching engine module 303 performs an operation of looking up the address table 306 on the incoming packet, thereby obtaining the direction information of the packet; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in association with the stream-id; if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downlink network interface to the uplink network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the guiding information of the packet; if the queue of the packet buffer 307 is nearly full, it is discarded.

The switching engine module 303 polls all packet buffer queues and may include two cases:

if the queue is from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the port send buffer is not full; 2) the queued packet counter is greater than zero; 3) obtaining a token generated by a code rate control module;

if the queue is not from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero.

The rate control module 308 is configured by the CPU module 304, and generates tokens for packet buffer queues from all downstream network interfaces to upstream network interfaces at programmable intervals to control the rate of upstream forwarding.

The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306, and configuration of the code rate control module 308.

Ethernet protocol conversion gateway：

As shown in fig. 4, the apparatus mainly includes a network interface module (a downlink network interface module 401 and an uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a rate control module 408, an address table 406, a packet buffer 407, a MAC adding module 409, and a MAC deleting module 410.

Wherein, the data packet coming from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the ethernet MACDA, the ethernet MACSA, the ethernet length or frame type, the video network destination address DA, the video network source address SA, the video network packet type, and the packet length of the packet meet the requirements, and if so, allocates a corresponding stream identifier (stream-id); then, the MAC deletion module 410 subtracts MACDA, MACSA, length or frame type (2byte) and enters the corresponding receiving buffer, otherwise, discards it;

the downlink network interface module 401 detects the sending buffer of the port, and if there is a packet, acquires the ethernet MACDA of the corresponding terminal according to the video network destination address DA of the packet, adds the ethernet MACDA of the terminal, the MACSA of the ethernet coordination gateway, and the ethernet length frame type, and sends the packet.

The other modules in the ethernet protocol gateway function similarly to the access switch.

A terminal:

the system mainly comprises a network interface module, a service processing module and a CPU module; for example, the set-top box mainly comprises a network interface module, a video and audio coding and decoding engine module and a CPU module; the coding board mainly comprises a network interface module, a video and audio coding engine module and a CPU module; the memory mainly comprises a network interface module, a CPU module and a disk array module.

1.3 devices of the metropolitan area network part can be mainly classified into 2 types: node server, node exchanger, metropolitan area server. The node switch mainly comprises a network interface module, a switching engine module and a CPU module; the metropolitan area server mainly comprises a network interface module, a switching engine module and a CPU module.

2. Video networking packet definition

2.1 Access network packet definition

The data packet of the access network mainly comprises the following parts: destination Address (DA), Source Address (SA), reserved bytes, payload (pdu), CRC.

As shown in the following table, the data packet of the access network mainly includes the following parts:

DA

SA

Reserved

Payload

CRC

wherein:

the Destination Address (DA) is composed of 8 bytes (byte), the first byte represents the type of the data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are 256 possibilities at most, the second byte to the sixth byte are metropolitan area network addresses, and the seventh byte and the eighth byte are access network addresses;

the Source Address (SA) is also composed of 8 bytes (byte), defined as the same as the Destination Address (DA);

the reserved byte consists of 2 bytes;

the payload part has different lengths according to different types of datagrams, and is 64 bytes if the datagram is various types of protocol packets, and is 32+1024 or 1056 bytes if the datagram is a unicast packet, of course, the length is not limited to the above 2 types;

the CRC consists of 4 bytes and is calculated in accordance with the standard ethernet CRC algorithm.

2.2 metropolitan area network packet definition

The topology of a metropolitan area network is a graph and there may be 2, or even more than 2, connections between two devices, i.e., there may be more than 2 connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the metro network address of the metro network device is unique, and in order to accurately describe the connection relationship between the metro network devices, parameters are introduced in the embodiment of the present application: a label to uniquely describe a metropolitan area network device.

In this specification, the definition of the Label is similar to that of the Label of MPLS (Multi-Protocol Label Switch), and assuming that there are two connections between the device a and the device B, there are 2 labels for the packet from the device a to the device B, and 2 labels for the packet from the device B to the device a. The label is classified into an incoming label and an outgoing label, and assuming that the label (incoming label) of the packet entering the device a is 0x0000, the label (outgoing label) of the packet leaving the device a may become 0x 0001. The network access process of the metro network is a network access process under centralized control, that is, address allocation and label allocation of the metro network are both dominated by the metro server, and the node switch and the node server are both passively executed, which is different from label allocation of MPLS, and label allocation of MPLS is a result of mutual negotiation between the switch and the server.

As shown in the following table, the data packet of the metro network mainly includes the following parts:

DA

SA

Reserved

label (R)

Payload

CRC

Namely Destination Address (DA), Source Address (SA), Reserved byte (Reserved), tag, payload (pdu), CRC. The format of the tag may be defined by reference to the following: the tag is 32 bits with the upper 16 bits reserved and only the lower 16 bits used, and its position is between the reserved bytes and payload of the packet.

Based on the characteristics of the video network, one of the core concepts of the embodiment of the application is provided, and the monitoring access server is applied to the video network, and can detect whether a first video stream of a target monitoring device is received or not when receiving a video calling signaling sent by an IE plug-in; if so, sending the first video stream to a sharing platform server; if not, returning a first error code to the Mserver through the shared platform server; if the sharing platform server receives the first video stream, the first video stream is converted into a second video stream of an Internet protocol, when the second video stream is sent to the IE plug-in unit, the sending state is detected, and if the sending is unsuccessful, a second error code is returned to the Mserver; the Mserver converts the first error code or the second error code into text type information which is convenient for a user to directly view, and returns the converted first text error information or the converted second text error information to the IE plugin, wherein the IE plugin displays the second video stream if receiving the second video stream, and displays the first text error information or the second text error information if receiving the first text error information or the second text error information. Therefore, whether the video stream is called successfully or not can be achieved, the result of calling the video stream can be known through the displayed content, and targeted operation can be performed according to the displayed result state, so that the problem that in the prior art, if the monitoring video cannot be called, a user repeatedly operates after waiting for a long time to crash a system is solved.

Example one

Referring to fig. 5, a flowchart illustrating steps of a method for calling a surveillance video according to an embodiment of the present application is shown, where the method is applied to a video network, where the video network includes a surveillance networking management scheduling platform server Mserver, a shared platform server communicatively connected to the Mserver, and a surveillance access server communicatively connected to the shared platform server, the surveillance access server is further communicatively connected to a plurality of surveillance devices in the internet, and the Mserver is further communicatively connected to a browser IE plug-in the internet.

Referring to fig. 6, the foregoing application environment diagram of the surveillance video retrieving method according to the embodiment of the present application is shown. The Mserver serves a video networking monitoring networking management scheduling platform in the video networking, the video networking monitoring networking management scheduling platform can use the Mserver as a background server, a front-end web interface in the Internet serves as a man-machine interaction interface, and the IE plug-in can run in the front-end web interface. Mserver can be connected to video networking and internet simultaneously, can communicate with IE plug-in components with internet protocol to video networking protocol and shared platform server communication, shared platform server can communicate through video networking protocol with control access server, control access server can with a plurality of supervisory equipment communication connection in the internet, Mserver, shared platform server and control access server all can be connected with video networking and internet simultaneously. Therefore, the IE plug-in running in the Internet can also be directly in communication connection with the sharing platform server, and further can directly receive the video stream sent by the sharing platform server.

The monitoring video calling method specifically comprises the following steps:

step 501, the IE plugin generates a first video calling signaling which accords with an internet protocol according to the received stream adjusting operation, and sends the first video calling signaling to the Mserver.

In practice, the IE plugin may generate a first video call signaling according to a received call flow operation of a user for a monitoring video list, where the monitoring video list may include device numbers of a plurality of monitoring devices, and the user may select a target device number through operations such as clicking, touching, and remote control, and then the IE plugin may generate the first video call signaling including the target device number according to a selection of the user, and send the first video call signaling to the Mserver. It should be noted that the first video call signaling may include a destination device number, and may also include a MAC address, an IP address, and the like of Mserve, which is not limited in this embodiment of the application.

In practical application, the first video call signaling may be signaling conforming to a sip (Session initiation protocol), the sip protocol is an application layer control protocol, and when the IE plug-in generates the first video call signaling conforming to the sip protocol, the first video call signaling conforming to the sip protocol may be encapsulated by using a TCP/IP protocol in an internet protocol, and then the first video call signaling encapsulated by using the TCP/IP protocol is sent to the Mserver.

Step 502, the Mserver converts the first video calling signaling into a second video calling signaling which accords with a video networking protocol, and sends the second video calling signaling to the sharing platform server.

In this embodiment of the application, because the IE plugin operates in the internet, the Mserver may receive the first video call signaling through the internet network card, may remove the TCP/IP protocol encapsulating the first video call signaling, and encapsulates the first video call signaling again using the video networking exchange protocol, thereby obtaining the second video call signaling, and may send the second video call signaling to the shared platform server through the video networking network card, and the shared platform server may receive the second video call signaling through the video networking network card, and may forward the second video call signaling to the monitoring access server through the video networking network card.

Through the

above steps

502 and 503, the first video call signaling sent by the IE plugin from the internet is successfully transmitted to the monitoring access server in the video network, thereby realizing the cross-network transmission of the first video call signaling.

Step 503, the monitoring access server receives the second video call signaling forwarded by the sharing platform server, requests a first video stream from a target monitoring device according to the second video call signaling, and detects whether the first video stream is received.

In the embodiment of the application, the monitoring access server can receive the second video calling signaling forwarded by the sharing platform server through a video networking network card of the monitoring access server, when the second video calling signaling is received, a video networking protocol can be adopted to analyze the second video calling signaling, a target equipment number can be extracted from the second video calling signaling, then in a plurality of connected monitoring equipment, the target monitoring equipment can be determined according to the target equipment number, a first video stream can be requested from the target monitoring equipment, and specifically, a video request signaling conforming to the internet protocol can be sent to the target monitoring equipment to request for monitoring video. In practical application, although the target monitoring device is connected to the monitoring access server, after the connection is established, the target monitoring device in the current state may not be able to successfully perform normal audio/video service communication with the monitoring access server for various reasons such as a network, the target monitoring device itself, and the like. Therefore, after requesting the first video stream from the target monitoring device, the monitoring access server may detect whether the first video stream returned by the target monitoring device for the video request signaling is received within a preset time.

If the first video stream is received, go to step 5031; if the first video stream is not received, go to step 5032.

Step 5031, sending the first video stream to the shared platform server.

The first video stream may be a video stream encapsulated by an internet protocol, and when receiving the first video stream, the monitoring access server may convert the first video stream into a first video stream conforming to a video networking protocol, and transmit the first video stream conforming to the video networking protocol to the shared platform server through a video networking network card.

Step 5032, a first error code is generated according to the current offline state of the target monitoring device or the current state of the video networking resource of the monitoring access server, and the first error code is sent to the shared platform server.

In practice, if the first video stream is not received within a preset time, it indicates that the target monitoring device cannot currently perform normal audio and video communication with the monitoring access server, in this case, the target monitoring device may be offline, or the monitoring access server itself may not have enough video networking resources to support receiving the first video stream of the target monitoring device, in order to obtain a reason that the first video stream is not successfully received, so that a user may troubleshoot problems afterwards, when the first video stream is not received, the monitoring access server may first detect whether the current state of the target monitoring device is an offline state, and if the current state is an offline state, generate the first error code according to the current offline state; if the state is the online state, the current state of the resources of the video networking of the monitoring access server is obtained, wherein the resources of the video networking can be understood as video transmission resources provided by the monitoring access server for each monitoring device performing the audio and video service, the state of the resources of the video networking is used for representing the current state of each resource of the video networking, and the monitoring access server can generate the first error code according to the current state of the resources of the video networking.

For example, if it is detected that the target monitoring device is offline, a first error code may be generated according to the current offline state, and the generated first error code may be "0001"; if the target monitoring device is detected to be online, the current state of the resources of the video network of the monitoring access server is further acquired, the current state of the resources of the video network can be divided into a full state and an unoccupied state, if the current state is the full state, the generated first error code can be '0002', and if the current state is the unoccupied state, the generated first error code can be '0003'.

After the first error code is generated, the monitoring access server may send the first error code to the shared platform server through the network card of the video network.

Step 504, when receiving the first error code sent by the monitoring access server, the shared platform server sends the first error code to the Mserver; or when the first video stream sent by the monitoring access server is received, converting the first video stream into a second video stream conforming to an internet protocol, and when the second video stream is sent to the IE plugin, detecting the state of sending the second video stream.

In this embodiment, the shared platform server may receive the first error code or the first video stream sent by the monitoring access server through a video networking network card. In practice, when receiving the first error code, the shared platform server may directly send the first error code to the Mserver through the network card of the video network of the shared platform server; when receiving the first video stream, the first video stream may be encapsulated into a video stream of an RTP (Real-time Transport Protocol) Protocol, and a TCP/IP packet header is added to the video stream of the RTP Protocol to obtain a second video stream, and then the second video stream may be sent to the IE plugin through an internet network card of the second video stream.

When the shared platform server sends the second video stream to the IE plugin, the sending of the second video stream may also be failed due to a network, the server itself, and the like, so that the shared platform server may detect a state of sending the second video stream when sending the second video stream, specifically, may detect whether the second video stream is successfully sent, if the second video stream is successfully sent, the shared platform server does not perform any operation, and if the second video stream is not successfully sent, it is determined that the state of sending the second video stream is a sending failure state, and then step 5041 is performed.

Step 5041, generating a second error code, and sending the second error code to the Mserver.

In practice, if the sharing platform fails to successfully transmit the second video stream, a second error code may be generated, where the second error code indicates that the sharing platform failed to successfully transmit the second video stream. For example, the second error code may be "4".

Step 505, the Mserver converts the first error code into a first text error message conforming to the sip protocol, or converts the second error code into a second text error message conforming to the sip protocol; and sending the first text error message or the second text error message to the IE plugin.

In the embodiment of the application, the Mserver can receive the first error code forwarded by the shared platform server through a video network card, or can receive the second error code generated and sent by the shared platform server through the video network card. If the received first error code is the first error code, the Mserver can convert the first error code into first text error information which accords with the sip protocol, and can send the first text error information to the IE plugin through the Internet network card of the Mserver. If the second error code is received, the Mserver can convert the second error code into second text error information which accords with the sip protocol, and can send the second text error information to the IE plug-in through the Internet network card of the Mserver. The first text error information and the second text error information are both sent to the IE plugin based on the Internet protocol, and when the IE plugin is packaged by the SIP protocol, the IE plugin can be read and analyzed conveniently.

For example, taking the case that the Mserver receives the first error code as an example, if the first error code is "0001", the monitoring access server may convert "0001" into the first text error message "offline" with the monitoring device.

In practice, the Mserver may convert the first error code into the first text error message according to a preset error code comparison table, or convert the second error code into the second text error message. The error code comparison table comprises a plurality of error codes and error information one-to-one corresponding to the error codes, wherein the error information is matched with a reason pointed by the error code. When the Mserver converts the first error code into the first text error message, the Mserver can find out the first error message corresponding to the first error code from the error code comparison table, and can convert the first error code into the first text error message matched with the first error message. Likewise, the second text error message may also be converted in this way. In practice, both the first text error message and the second text error message may be error messages conforming to the text format.

For example, taking the case that the Mserver receives the first error code, if the first error code is "0001" and is generated by the monitoring access server when the monitoring access server detects that the target monitoring device is offline, according to the error code comparison table, the error information corresponding to "0001" is "monitoring device offline", and the "monitoring device offline" matches with the "current offline state of the target monitoring device", the "0001" may be converted into the first text error information matching with the "monitoring device offline", and the first text error information may be information that is consistent with the meaning of "monitoring device offline", such as "monitoring device offline", or may be information completely consistent with "monitoring device offline".

For another example, if the Mserver receives the second error code, the second error code is "4", and the shared platform server detects that the state of sending the second video stream is the state of sending failure, the error information corresponding to the "4" is the "shared platform sending error" according to the error code comparison table, and the "shared platform sending error" matches the "state of sending the second video stream as the state of sending failure", the "4" may be converted into the second text error information matching the "shared platform sending error", and the second text error information may be information that is consistent with the meaning of the "shared platform sending error", such as "shared platform sending failure", or may be information that is completely consistent with the "shared platform sending error".

In practice, the error codes and the corresponding error information in the error code comparison table can be set by a user according to actual conditions, and the user can add some error codes or delete some error codes, so that the converted text error information can be improved and is convenient for the user to identify and understand.

Step 506, if the IE plugin receives the second video stream, playing the second video stream; and if the first text error message or the second text error message is received, displaying the first text error message or the second text error message.

In the embodiment of the application, if the IE plugin receives the second video stream sent by the shared platform server, the second video stream may be decoded and played; if the received first text error message or the second text error message sent by the Mserver, the first text error message or the second text error message can be displayed. In practice, the IE plug-in may play the second video stream on a preset play window, and may display the first text error information or the second text error information in the preset play window. In this way, the user may view the called second video stream in the playing window, or may view the displayed first text error information or the second text error information in the playing window. Therefore, different results of the surveillance video calling can be displayed through the same display medium, if a user sees a video picture in the playing window, the user determines that the video stream is successfully called, and if the user sees text error information in the playing window, the user determines that the video stream is failed to be called, so that the user can intuitively know the result of the surveillance video calling through the picture displayed in the playing window.

In the embodiment of the application, the first text error information and the second text error information can be error information conforming to a text format, and the IE plug-in displays text information conforming to the language habit of the user instead of error codes, so that the user can more intuitively acquire the reason when the video stream is not called, and further can take targeted measures.

For example, the first text error information is "monitoring device offline", a text word of "monitoring device offline" is displayed in the playing window, and when the user sees the text information, the user can know that the video calling is unsuccessful due to the fact that the target monitoring device is offline, and at the moment, the user can select to call the monitoring videos of other monitoring devices or inform the target monitoring device of being online to complete the calling of the monitoring videos, so that the problem that the system is broken down due to the fact that the user repeatedly calls the monitoring videos of the target monitoring device can be avoided, the user does not need to spend a large amount of time to troubleshoot the problem, and user experience is improved.

According to the embodiment of the application, after a user initiates a first video calling signaling through an IE plug-in, if the calling of a video stream is successful, a video picture collected by a target monitoring device is played in a playing window of the IE plug-in, and if the calling of the video stream is failed, text error information corresponding to the loss of the calling video is displayed in the playing window of the IE plug-in. Therefore, the user can display the picture according to the video picture or the text error information appearing in the playing window of the IE plug-in, the result of calling the monitoring video is timely obtained, and the user experience is improved. Furthermore, the problem that in the prior art, after a user sends a video calling signaling through an IE plug-in, if the calling failure is delayed and the played video is not seen, the user can not acquire the result of the calling failure and can not acquire the reason of the calling failure, so that the user has to repeat the video calling operation to cause system crash is solved, and meanwhile, the technical problem that the user needs to spend a large amount of time on troubleshooting due to the fact that the user cannot acquire the reason of the calling failure is solved.

The embodiment of the application shows another monitoring video calling method, which is applied to a video network, the video network includes a monitoring network management scheduling platform server Mserver, a shared platform server in communication connection with the Mserver, and a monitoring access server in communication connection with the shared platform server, the monitoring access server is further in communication connection with a plurality of monitoring devices in the internet, and the Mserver is further in communication connection with a browser IE plug-in the internet, and specifically includes the following steps:

step 701, the IE plugin generates a first video calling signaling which accords with an internet protocol according to the received stream adjusting operation, and sends the first video calling signaling to the Mserver.

In the embodiment of the application, when a user needs to call a monitoring video, the user can open a browser through an intelligent device such as a Personal Computer (PC) or a mobile phone, and input website information corresponding to an IE plugin, so as to enter an interface where the IE plugin is located, and further perform stream-adjusting operation on the IE plugin.

Step 702, the Mserver converts the first video calling signaling into a second video calling signaling which conforms to a video networking protocol, and sends the second video calling signaling to the shared platform server.

In the embodiment of the application, the Mserver can receive a first video calling signaling through an Internet network card, convert the first video calling signaling into a second video calling signaling of a video networking protocol when receiving the first video calling signaling, and can send the second video calling signaling to the shared platform server through the video networking network card.

Step 703, the monitoring access server receives the second video call signaling forwarded by the sharing platform server, requests a first video stream from a target monitoring device according to the second video call signaling, and detects whether the first video stream sent by the target monitoring device is received.

In this embodiment of the application, after the second video call signaling is generated by Mserver, the second video call signaling may be forwarded to the monitoring access server via the shared platform server, and when the monitoring access server receives the second video call signaling, the target device number in the second video call signaling may be analyzed, and a target device address corresponding to the target device number may be searched in a pre-stored device list according to the target device number, so that a monitoring video may be requested from a target monitoring device corresponding to the target device address according to the determined target device address, and it may be detected whether the first video stream sent by the target monitoring device is received within a preset time, for example, within 5 seconds. If the first video stream is received, go to step 7031; if the first video stream is not received, go to step 7032.

Step 7031, sending the first video stream to the shared platform server.

Step 7032, a first error code is generated according to the current offline state of the target monitoring device or the current state of the resources of the video networking of the monitoring access server, and the first error code is sent to the shared platform server.

As an optional implementation manner of the embodiment of the present application, as shown in fig. 7, a flowchart of the sub-step of this step 7032 is shown, where the monitoring access server is configured with a database, and step 7032 may specifically include the following sub-steps:

sub-step a1, the monitoring access server detecting whether the target monitoring device is offline.

In this embodiment, the monitoring access server may send a heartbeat packet to the target monitoring device to detect whether the target monitoring device is offline, and if the heartbeat reply signaling of the target monitoring device is not received, it may be determined that the target monitoring device is offline, and if the heartbeat reply signaling of the target monitoring device is received, it may be determined that the target monitoring device is online.

The rotor step a2 is performed when the target monitoring device is determined to be offline, and the rotor step a3 is performed when the target monitoring device is determined to be online.

Sub-step a2, obtaining multiple resource change data corresponding to the identification information of the target monitoring device from the database, and generating the first error code according to the resource change data closest to the current time.

In this embodiment of the present application, the offline state of the target monitoring device may include reasons that the target monitoring device quits the monitoring access server, the target monitoring device fails to log in the monitoring access server, and the target monitoring device is removed by the monitoring access server, and in order to obtain a specific offline reason, so as to facilitate a user to troubleshoot a problem afterwards, the monitoring access server in this embodiment of the present application may correspondingly generate a resource change data when the target monitoring device is offline, and write the resource change data into the database, where the resource change data may include identification information that identifies the target monitoring device, such as a device number, a device address, and the like. In this way, when the monitoring access server detects that the target monitoring device is offline, the monitoring access server may further search a plurality of resource change data corresponding to the identification information of the target monitoring device in the database, where the identification information may be a target device number of the target monitoring device.

In practice, the target monitoring device may correspond to a plurality of resource change data, and if the target monitoring device performs 4 operations of exiting the monitoring access server within a period of time, 4 resource change data may be correspondingly generated, and in order to obtain the reason that the target monitoring device is currently offline, the monitoring access server may extract the resource change data closest to the current time from the plurality of resource change data found, and generate the first error code according to the resource change data closest to the current time.

For example, taking the target device number of the target monitoring device as 0025 as an example, the monitoring access server searches 4 pieces of resource change data corresponding to 0025 in the database. If the resource change data closest to the current time extracted from the 4 pieces of resource change data is "0025-login failure", which indicates that the target monitoring device has not successfully logged in to the monitoring access server, the first error code generated according to the "0025-login failure" may be "0 x 000C". If the resource change data closest to the current time is extracted from the 4 resource change data and is "0025-offline", which indicates that the target monitoring device is offline for the network reason, the first error code generated by the "0025-offline" may be "0 x 000E". If the resource change data closest to the current time extracted from the 4 pieces of resource change data is "0025-unresponsive," which indicates that the target monitoring device cannot normally respond to the monitoring access server (at this time, the heartbeat packet cannot be returned, which is also regarded as an offline state), the first error code generated according to "0025-unresponsive" may be "0 x 000B".

And a substep a3, acquiring the current state of the own video network resource, and generating the first error code according to the current state of the video network resource.

In the embodiment of the application, when it is detected that the target monitoring device is online, it indicates that the target monitoring device can send a first video stream, but cannot receive the first video stream due to the reason of the monitoring access server, the monitoring access server may further detect the state of the own video networking resource, specifically, it may determine the current state of the video networking resource by detecting whether the video sending port resource of the monitoring access server is fully occupied, if the video sending port resource is fully occupied, it indicates that the monitoring access server cannot receive the first video stream due to insufficient port resource, it may determine that the current state of the video networking resource is a state of insufficient video networking resource, and may generate a first error code according to the state of insufficient video networking resource; if the video sending port resource is not occupied, the port resource is not enough, if the first video stream is still not received, the current video networking resource state can be determined to be the video networking resource sufficient state, and under the condition, a first error code representing the video code stream error can be generated.

For example, when the current internet-of-view status of the monitoring access server is acquired as the state of insufficient internet-of-view resources, the generated first error code may be "0 x 0002". When the current internet-of-view status is a status with sufficient internet-of-view resources, the generated first error code may be "0 x 000B", which may indicate a code stream error of the first video stream, such that the monitoring access server cannot receive the error code.

Through the substeps a1 to a3, the reason that the monitoring access server does not receive the first video stream can be refined, so that a user can conveniently know the problem directly according to the first error code, and the time for the user to check the problem can be shortened.

Step 704, when receiving the first error code sent by the monitoring access server, the shared platform server sends the first error code to the Mserver; or when the first video stream sent by the monitoring access server is received, converting the first video stream into a second video stream conforming to an internet protocol, and when the second video stream is sent to the IE plugin, detecting the state of sending the second video stream.

In the embodiment of the application, if the shared platform server receives the first error code, the shared platform server can directly forward the first error code to the Mserver; if the received video stream is the first video stream, the first video stream can be converted into a second video stream conforming to the internet protocol, and the second video stream can be directly sent to the IE plugin without being passed through the Mserver. When the second video stream is transmitted, the state of transmitting the second video stream may also be detected, if it is detected that the second video stream is not successfully transmitted, it is determined that the state of transmitting the second video stream is a state of failed transmission, step 7041 is performed, and if it is detected that the second video stream is successfully transmitted, step 705 is performed.

Step 7041, a second error code is generated, and the second error code is sent to the Mserver.

In practice, the reasons for the failure of the shared platform server to send the second video stream are also various, in order to obtain detailed reasons for the failure of sending so that a user can troubleshoot problems afterwards, the method and the device for sending the second video stream can further detect the reason for the state causing the failure of sending, generate a second error code conforming to the video networking protocol according to the reason for the state causing the failure of sending, and then send the second error code to the Mserver through the video networking card of the device.

As an alternative embodiment of the present application, as shown in fig. 8, a flowchart of sub-steps of this step 7041 is shown, and this step 7041 may specifically include the following sub-steps:

sub-step b1, the shared platform server determines whether the IP address and port of the IE plugin is detected.

In practice, to successfully send the second video stream to the IE plugin, the IP address and the port of the IE plugin need to be obtained, in this embodiment of the present application, if the IP address and the port of the IE plugin are detected, it indicates that a condition for successfully sending the second video stream is met, and then step b2 is performed; if the IP address and port of the IE plugin cannot be detected, step b3 is followed.

And a sub-step b2 of detecting the TCP connection status of the IE plug-in and generating a second error code corresponding to the state that the TCP connection status is failed when the TCP connection status is failed.

In practice, if the shared platform server can obtain the IP address and the port of the IE plugin, it further detects whether a normal TCP connection can be established with the IE plugin, and if not, it can be determined that the TCP connection state is a failed state, and a second error code can be generated corresponding to the TCP connection state being the failed state, for example, the second error code may be "0 x 0066".

In practice, if the normal TCP connection cannot be established with the IE plug-in, the sharing platform may further detect whether there is enough virtual terminal number resource to support sending of the second video stream, in the internet of view, one virtual terminal number may correspond to each video stream, and if there is not enough virtual terminal number resource, a second error code conforming to the internet of view protocol may be generated according to "lack of virtual terminal number resource", for example, the second error code may be "0 x 0069". If the virtual terminal number resources are sufficient, the shared platform server can be set to continue to detect other information according to the actual situation, such as code stream information of the second video stream, and the like.

Sub-step b3, generating the second error code according to the result of not detecting the IP address and/or the port.

In practice, if the shared platform server cannot acquire the IP address and the port of the IE plugin, a second error code may be generated according to the result that the IP address and the port of the IE plugin cannot be acquired, and in practice, if the shared platform server determines that the acquired IP address of the IE plugin is not acquired, a second error may be generated according to the result that the IP address cannot be acquired, for example, the second error code may be "0 x 0067". If it is determined that the port of the IE plugin is not obtained, a second error may be generated according to the result of the port being not obtained, for example, the second error code may be "0 x 0068".

Through the steps b1 to b3, the reason that the sharing platform server does not successfully send the second video stream can be refined, so that a user can conveniently know the problem directly according to the second error code, and the time for the user to check the problem can be shortened.

Step 705, the Mserver converts the first error code into a first text error message conforming to a sip protocol, or converts the second error code into a second text error message conforming to the sip protocol; and sending the first text error message or the second text error message to the IE plugin.

In the embodiment of the application, the Mserver may convert the first error code into the first text error message according to the sip protocol based on a pre-stored error code comparison table, and the pre-stored error code comparison table may be pre-set according to an actual situation and pre-stored in the Mserver, as shown in table 1 below, an error code comparison table according to the embodiment of the application is shown, and in table 1 below, the error code comparison table includes a plurality of error codes and error messages corresponding to each error code, where the error messages may represent reasons pointed when the error codes are generated. Whether the first error code or the second error code is received, the Mserver can determine corresponding error information according to the error code comparison table and generate first text error information or second text error information matched with the error information. In practice, the first error code and the second error code may adopt different encoding formats, so that the first error code may be characterized as being generated by the monitoring access server and the second error code is generated by the shared platform server.

For example, as shown in table 1, the first error code received by the Mserver is "0 x 000C", the first error code is generated by the monitoring access server, and the reason for generating the first error code is that the target monitoring device fails to log in the monitoring access server, and then the error information corresponding to "0 x 000C" is found in the error code comparison table as "monitoring device on the monitoring access server fails to log in", and the generated first error text information may also be "monitoring device on the monitoring access server fails to log in", which it is seen that the error information restores the reason for receiving the first video stream, which is represented by the first error code, and thus the first error text information directly represents the reason for receiving the first video stream fails.

Error code	Error information
		0x0003	Monitoring that the access server has not had such monitoring resources
0x0002	Monitoring access server video networking resources being occupied
		0x000B	The video stream is failed to open, and the monitoring source equipment does not send the video stream
0x000C	Monitoring device login failure on access server
		0x000D	Monitoring access serverThe national standard lower platform is off-line
0x000E	Monitoring whether the channel of the national standard lower platform of the access server is off-line
		4	Shared platform server send errors
0x0065	Shared platform server detects that video stream already exists
		0x0066	Shared platform server detection of TCP connection failure
0x0067	Shared platform server detection of IP unreachability
		0x0068	Shared platform server detection port unreachable
0x0069	Shared platform server has no idle virtual terminal resource

TABLE 1

Step 706, if the IE plugin receives the second video stream, playing the second video stream; and if the first text error message or the second text error message is received, displaying the first text error message or the second text error message.

The process of step 706 is similar to that of step 506, and is not described herein again.

As an optional implementation manner of this embodiment of this application, as shown in fig. 9, after the IE plugin plays the second video stream, the method may further include the following steps:

and 707, detecting, by the monitoring access server, a state of receiving the first video stream at every preset first time interval, if it is detected that the state is a video reception failure state, obtaining state information corresponding to the video reception failure state, generating a third error code based on the state information, and sending the third error code to the shared platform server.

In this embodiment of the application, after the IE plugin successfully plays the second video stream, the monitoring access server may further detect, every preset first time, 1 time of obtaining the state of the first video stream from the target monitoring device, where the preset first time may be set according to an actual situation, for example, 20 seconds, if the first video stream is not detected, it indicates that the first video stream is interrupted, it is determined that the state of receiving the first video stream is a state of video reception failure, and then, state information corresponding to the state of video reception failure may be obtained, and in practice, when the first video stream is interrupted, reasons that may be included are: the target monitoring device is suddenly offline, the network has errors, and the like, and when the errors occur, the error information can be correspondingly generated, so that the monitoring access server of the embodiment of the application can determine a specific reason by acquiring the state information, and then generate a third error code according to the state information.

Illustratively, if the acquired state information is "dropped", it indicates that the target monitoring device is suddenly dropped so that the first video stream cannot be transmitted, and the correspondingly generated third error code may be "485"; if the acquired status information is "network busy", which indicates that the remaining bandwidth of the current internet network is not enough to successfully transmit the first video stream, the correspondingly generated third error code may be "500".

Step 708, the shared platform server sends the third error code to the Mserver.

In this embodiment of the application, the third error code is a third error code of the video networking protocol, and the shared platform server may send the third error code to the Mserver through a video networking network card.

Step 709, the Mserver converts the third error code into a third text error message conforming to the sip protocol, and sends the third text error message to the IE plugin.

The procedure of step 709 is similar to that of step 705, and will not be described in detail here.

Step 710, the IE plugin displays the third text error message.

In the embodiment of the application, when the IE plugin receives the third text error message sent by the Mserver, the third text error message is displayed, in practice, the third text error message can be displayed on the paused picture, and the third text error message can also be displayed on the black screen picture, so that a user can obtain the reason of picture pause or black screen through the third text error message, the problem is conveniently checked by the user, and the user experience is improved.

As an optional example of the embodiment of the present application, the internet further includes a third party platform server, and in step 707, the obtaining of the state information corresponding to the state of the video reception failure may specifically include the following sub-steps:

and a substep c1, when the first video stream is forwarded to the monitoring access server by the third-party platform server, the monitoring access server sends a state acquisition signaling conforming to the internet protocol to the third-party platform server.

In practice, the monitoring access server may be connected to a plurality of monitoring devices in the internet through a third-party platform server, so that when the first video stream of the target monitoring device is obtained, the first video stream may be forwarded to the monitoring access server through the third-party platform server. The monitoring access server may further detect whether the first video stream passes through the third-party platform server when detecting that the first video stream is not received. If the current state of the target monitoring equipment and/or the current state of the target monitoring equipment is/are not the same as the current state of the target monitoring equipment, a state acquisition signaling which conforms to an internet protocol can be sent to the third-party platform server to search the reason why the first video stream is interrupted from the third-party platform server; if the third-party platform server does not receive the state acquisition signaling, the communication abnormality of the third-party platform server can be represented, and complex information cannot be returned.

In this sub-step c1, the monitoring access server may detect whether a reply message returned by the third party platform server is received within a preset third time, if yes, the step c2 is performed; if not, step c3 is rotated. Wherein the third time may be set by the user, for example, to 5 seconds.

Sub-step c2, determining the reply information as the status information; the reply information comprises offline reply information and/or server busy reply information and/or response overtime reply information.

In this embodiment of the application, the third-party platform server may receive the state acquisition signaling, which indicates that the communication between the third-party platform server and the monitoring access server is normal, and may determine that the reason for the interruption of the first video stream is that an error occurs in a communication path between the target monitoring device and the third-party platform server, and the third-party platform server may simultaneously detect the current state of the target monitoring device and the current state of itself, or may only detect the current state of the target monitoring device or only detect the current state of itself. If the target monitoring equipment is detected to be offline, offline reply information can be correspondingly generated; if the first video stream cannot be received due to the fact that the network of the server is busy, server busy reply information can be correspondingly generated; if the response timeout of the self server for receiving the first video stream is detected, response timeout reply information can be correspondingly generated.

The monitoring access server may determine the reply information as the status information when receiving the reply information, and further may generate a third error code based on the reply information. In practice, if the reply message includes the three reply messages, three sub-error codes can be generated accordingly, so that the third error code includes three sub-error codes.

For example, when the reply message is a drop-line reply message, the corresponding third error code may be "404"; when the reply message is a busy reply message, the corresponding third error code may be "486"; when the reply message is the response message of the server responding to the timeout reply message, the corresponding third error code may be "408". If the third error code includes the three reply messages, the third error code includes three sub-error codes: "404", "486", and "408".

In practice, the reply message may also include information generated by the third party platform server according to other detected errors related to the transmission of the first video stream, for example, if the third party platform detects an error in the loop, the reply message may include a "loop error reply message", and the monitoring access server may generate a third error code based on the "loop error reply message", for example, may be "482".

And a substep c3, determining the information that the third party platform server has the abnormity as the state information.

In this embodiment of the application, within a preset third time, no reply message of the third-party platform server is received, which indicates that the third-party platform server does not receive the state acquisition signaling, and it may be determined that the first video stream is interrupted due to a communication abnormality of the third-party platform server, and it may be determined that the information that the third-party platform server is abnormal is state information, and then a third error code is generated according to the state information, for example, the third error code may be "491".

As another optional example of the embodiment of the present application, as shown in fig. 10, after the IE plugin plays the second video stream, the method may further include the following steps:

step 711, the shared platform server detects the communication state with the IE plugin every preset second time interval, and detects whether the first video stream is received; if the communication state is a disconnected state, generating a fourth error code; if the first video stream is not received, generating a fifth error code; and sending the fourth error code and/or the fifth error code to the Mserver.

In this embodiment of the application, after the IE plugin successfully plays the second video stream, the shared platform server may detect, every 1 time, a communication state between itself and the IE plugin and whether the second video stream is received. Wherein the second time may also be set by the user, and in practice, may coincide with the first time, for example, 20 seconds; specifically, the shared platform server may send a heartbeat packet to the IE plugin at every preset second time, and if the IE plugin replies a heartbeat signaling, it is determined that communication with the IE plugin is normal; if the IE plugin does not reply the heartbeat packet, it is determined that the communication state with the IE plugin is a disconnection state, and a fourth error code may be generated. And, detecting whether a first video stream arrives at a preset second time interval, and if the first video stream is not detected, indicating that an error occurs in a video communication path between the monitoring access server and the shared platform server, correspondingly generating a fifth error code.

Illustratively, the fifth error code may be "702" if the first video stream is not detected, and the fourth error code may be "701" if the communication status with the IE plugin is detected as a disconnected status.

Step 712, when receiving the fourth error code and/or the fifth error code, the Mserver converts the fourth error code into a fourth text error message conforming to the sip protocol, and/or converts the fifth error code into a fifth text error message conforming to the sip protocol; and sending the fourth text error information and/or the fifth text error information to the IE plugin.

The process of step 712 is similar to that of step 705, and will not be described in detail here.

Step 713, the IE plugin displays the fourth text error information and/or the fifth text error information.

The procedure of step 713 is similar to that of step 710 and will not be described in detail herein.

As an optional example of the embodiment of the present application, step 701 may specifically include the following: and after the IE plug-in sends the first video calling signaling, generating stream regulation initiating information and displaying the stream regulation initiating information.

In this optional embodiment, the IE plug-in may generate the stream adjustment initiation information after the first video call signaling is sent, where the stream adjustment initiation information may represent a state in which the IE plug-in has initiated the call of the video stream, and display the stream adjustment initiation information. If the stream adjustment initiating information is "signaling establishment waiting video stream", the displayed information is also "signaling establishment waiting video stream", and the user can determine the current stream adjustment progress according to the displayed information. In practice, the stream adjustment initiation information may be displayed in a preset play window.

In the embodiment of the application, when the user initiates the first video call signaling through the IE plugin, the call initiation information can be displayed, so that the user can know the current call progress. When the monitoring access server detects that the first video stream of the target monitoring equipment is not received, further detecting whether the target monitoring equipment is offline, and if the target monitoring equipment is offline, generating a first error code according to the resource change data closest to the current time; when receiving a first video stream, the sharing platform server converts the first video stream into a second video stream, further detects the IP address and port of the IE plugin, the TCP connection state with the IE plugin and the like, generates a second error code according to the detection result, and finally enables the first text error information or the second text error information displayed in the IE plugin to reflect the detailed reason of failure in receiving the first video stream or the detailed reason of failure in sending the second video stream, so that the time of troubleshooting of a user can be further shortened, and the operation efficiency is improved.

Furthermore, after the second video stream is played, the monitoring access server can also detect the receiving state of the received first video stream at regular time, and the sharing server can also detect the sending state of the second video stream at regular time, so that in the process of calling the video stream of the target monitoring device, the receiving state of the monitoring access server and the sending state of the sharing server can be monitored at regular time, and thus, when the video is interrupted, a user can also know the reason of the video interruption according to the displayed third text error information, fourth text error information or fifth text error information, so that the user can solve the problem in a targeted manner according to the reason of the video interruption, and the time for the user to check the problem is further shortened. Therefore, the problem that in the prior art, once the playing picture is interrupted or the screen is blacked, a user cannot know the reason and needs to spend a large amount of time to investigate the problem is solved.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the embodiments. Further, those skilled in the art will also appreciate that the embodiments described in the specification are presently preferred and that no particular act is required of the embodiments of the application.

Example two

Referring to fig. 11, a surveillance video retrieval system according to an embodiment of the present application is shown, where the system is applied to a video network, where the video network includes a surveillance networking management scheduling platform server Mserver, a shared platform server communicatively connected to the Mserver, and a surveillance access server communicatively connected to the shared platform server, where the surveillance access server is communicatively connected to a plurality of surveillance devices in the internet, and the Mserver is further communicatively connected to a browser IE plug-in the internet; the system may specifically comprise the following modules:

a video stream signaling module 1101, located in the IE plugin, and configured to generate a first video call signaling according to an internet protocol according to a received stream call operation, and send the first video call signaling to the Mserver;

a signaling conversion module 1102, located in the Mserver, configured to convert the first video call signaling into a second video call signaling that conforms to a video networking protocol, and send the second video call signaling to the shared platform server;

a video stream detection module 1103, located in the monitoring access server, and configured to receive the second video call signaling forwarded by the sharing platform server, request a first video stream from a target monitoring device according to the second video call signaling, and detect whether the first video stream is received;

a video stream forwarding module 1104, located at the monitoring access server, and configured to send the first video stream to the sharing platform server when receiving the first video stream;

a first error code generating module 1105, located in the monitoring access server, and configured to generate a first error code according to the current offline state of the target monitoring device or the current state of the video network resource of the monitoring access server if the first video stream is not received, and send the first error code to the shared platform server;

an error code forwarding module 1106, located in the shared platform server, and configured to send the first error code to the Mserver when the first error code sent by the monitoring access server is received;

a video sending state detection module 1107, located in the shared platform server, and configured to convert the first video stream into a second video stream conforming to an internet protocol when receiving the first video stream sent by the monitoring access server, and detect a state of sending the second video stream when sending the second video stream to the IE plugin;

a second error code generation module 1108, located in the shared platform server, and configured to generate a second error code and send the second error code to the Mserver if the status is a sending failure status;

an error code conversion module 1109, located in the Mserver, configured to convert the first error code into a first text error message conforming to a session initiation sip protocol, or convert the second error code into a second text error message conforming to a sip protocol; sending the first text error message or the second text error message to the IE plug-in;

a display module 1110, located in the IE plugin, and configured to play the second video stream if the second video stream is received; and if the first text error message or the second text error message sent by the Mserver is received, displaying the first text error message or the second text error message.

Optionally, the system may further include:

the first routing inspection module is positioned at the monitoring access server and used for detecting the state of receiving the first video stream at every preset first time, if the state is detected to be a video receiving failure state, state information corresponding to the video receiving failure state is acquired, a third error code is generated based on the state information, and the third error code is sent to the sharing platform server; the shared platform server is used for sending the third error code to the Mserver;

the error code conversion module 1109 is further configured to convert the third error code into third text error information conforming to the sip protocol, and send the third text error information to the IE plugin;

the display module 1110 is further configured to display the third text error message.

Optionally, the system may further include:

the second routing inspection module is positioned on the shared platform server and used for detecting the communication state of the IE plug-in at preset second time intervals and detecting whether the second video stream is received or not; if the communication state is a disconnected state, generating a fourth error code; if the second video stream is not received, generating a fifth error code; sending the fourth error code and the fifth error code to the Mserver;

the error code conversion module 1109 is further configured to, when the fourth error code and/or the fifth error code is received, convert the fourth error code into fourth text error information conforming to the sip protocol, and/or convert the fifth error code into fifth text error information conforming to the sip protocol; and sending the fourth text error information and/or the fifth text error information to the IE plugin;

the display module 1110 is further configured to display the fourth text error message and/or the fifth text error message.

Optionally, the video stream signaling module may further include:

the flow regulation initiating information generating submodule is used for generating flow regulation initiating information after the first video calling signaling is sent;

the display module 1110 is further configured to display the stream adjustment initiating information.

Optionally, the monitoring access server is configured with a database, and the first error code generation module may include the following units:

the off-line detection unit is used for detecting whether the target monitoring equipment is off-line or not;

a first generating unit, configured to, if the target monitoring device is offline, obtain, from the database, a plurality of resource change data corresponding to the identification information of the target monitoring device, and generate the first error code according to the resource change data closest to the current time;

the second generation unit is used for acquiring the current video network resource state of the target monitoring equipment if the target monitoring equipment is online and generating the first error code according to the current video network resource state;

the second error code generation module may include the following units:

a judging unit, configured to judge whether the IP address and the port of the IE plugin are detected;

a third generating unit, configured to detect a TCP connection status of the IE plugin if the IP address and the port of the IE plugin are detected, and generate a second error code corresponding to a state in which the TCP connection status is failed when the TCP connection status is in a failed state;

a fourth generating unit, configured to generate the second error code according to a result that the IP address and/or the port are not detected if the IP address and the port of the IE plugin are not detected.

Optionally, the internet further includes a third party platform server, and the first generating unit further includes the following sub-modules:

a signaling sending submodule, configured to send, by the monitoring access server, a state acquisition signaling conforming to an internet protocol to the third-party platform server when the first video stream is forwarded to the monitoring access server by the third-party platform server;

the first confirmation submodule is used for determining reply information returned by the third-party platform server aiming at the state acquisition signaling as the state information if the reply information is received within a preset third time; the reply information comprises offline reply information and/or server busy reply information and/or response overtime reply information;

and the second confirmation submodule determines that the information that the third-party platform server is abnormal is the state information if the reply information returned by the third-party platform server is not received within a preset third time.

As for the embodiment of the surveillance video retrieval system, since it is basically similar to the embodiment of the surveillance video retrieval method, the description is relatively simple, and relevant points refer to part of the description of the embodiment of the surveillance video retrieval method.

As will be appreciated by one of skill in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The foregoing describes a surveillance video retrieval method and a surveillance video retrieval system provided by the present application, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the description of the foregoing embodiments is only used to help understand the method and the core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A monitoring video calling method is applied to a video network, wherein the video network comprises a monitoring network management scheduling platform server Mserver, a shared platform server in communication connection with the Mserver, and a monitoring access server in communication connection with the shared platform server, the monitoring access server is also in communication connection with a plurality of monitoring devices in the Internet, and the Mserver is also in communication connection with a browser IE plug-in the Internet; the method comprises the following steps:

the Mserver converts the first error code into a first text error message which accords with an initial session sip protocol, or converts the second error code into a second text error message which accords with the sip protocol; sending the first text error message or the second text error message to the IE plug-in;

2. The method of claim 1, wherein after the step of playing the second video stream, the method further comprises:

detecting a state of receiving the first video stream at every preset first time interval by the monitoring access server, if the state is detected to be a video receiving failure state, acquiring state information corresponding to the video receiving failure state, generating a third error code based on the state information, and sending the third error code to the sharing platform server;

the shared platform server sends the third error code to the Mserver;

the Mserver converts the third error code into third text error information which accords with a sip protocol, and sends the third text error information to the IE plugin;

and the IE plug-in displays the third text error message.

3. The method according to claim 1 or 2, wherein after the step of playing the second video stream, the method further comprises:

the shared platform server detects the communication state with the IE plug-in every preset second time interval, and detects whether the first video stream is received or not; if the communication state is a disconnected state, generating a fourth error code; if the first video stream is not received, generating a fifth error code; and sending the fourth error code and/or the fifth error code to the Mserver;

when the Mserver receives the fourth error code and/or the fifth error code, converting the fourth error code into fourth text error information conforming to the sip protocol, and/or converting the fifth error code into fifth text error information conforming to the sip protocol; and sending the fourth text error information and/or the fifth text error information to the IE plugin;

and the IE plug-in displays the fourth text error information and/or the fifth text error information.

4. The method of claim 1, wherein the monitoring access server is configured with a database, and wherein generating the first error code according to the current offline status of the target monitoring device or the current state of the resources of the video networking of the monitoring access server comprises:

the monitoring access server detects whether the target monitoring equipment is off-line;

if the target monitoring equipment is offline, acquiring a plurality of resource change data corresponding to the identification information of the target monitoring equipment from the database, and generating the first error code according to the resource change data closest to the current time;

if the target monitoring equipment is online, acquiring the current video network resource state of the target monitoring equipment, and generating the first error code according to the current video network resource state;

the generating a second error code includes:

the shared platform server judges whether the IP address and the port of the IE plug-in are detected or not;

if so, detecting the TCP connection state of the IE plug-in, and generating a second error code corresponding to the state that the TCP connection state is failed when the TCP connection state is the failed state;

and if not, generating the second error code according to the result that the IP address and/or the port are not detected.

5. The method according to claim 2, wherein the internet further comprises a third-party platform server, and the obtaining of the status information corresponding to the status of the video reception failure comprises:

when the first video stream is forwarded to the monitoring access server by the third-party platform server, the monitoring access server sends a state acquisition signaling conforming to an internet protocol to the third-party platform server;

if the reply information returned by the third-party platform server aiming at the state acquisition signaling is received within the preset third time, determining the reply information as the state information; the reply information comprises offline reply information and/or server busy reply information and/or response overtime reply information;

and if the reply information returned by the third-party platform server is not received within a preset third time, determining that the information of the third-party platform server with the abnormality is the state information.

6. The method of claim 1, wherein sending the first video invocation signaling to the Mserver comprises:

and after the IE plug-in sends the first video calling signaling, generating stream regulation initiating information and displaying the stream regulation initiating information.

7. A monitoring video calling system is applied to a video network, wherein the video network comprises a monitoring network management scheduling platform server Mserver, a shared platform server in communication connection with the Mserver, and a monitoring access server in communication connection with the shared platform server, the monitoring access server is in communication connection with a plurality of monitoring devices in the Internet, and the Mserver is also in communication connection with a browser IE plug-in the Internet; the system comprises:

the video stream forwarding module is positioned in the monitoring access server and used for sending the first video stream to the sharing platform server if the first video stream is received;

the display module is positioned in the IE plug-in and is used for playing the second video stream if the second video stream is received; and if the first text error message or the second text error message sent by the Mserver is received, displaying the first text error message or the second text error message.

8. The system of claim 7, further comprising:

the error code forwarding module is further configured to convert the third error code into third text error information conforming to an sip protocol, and send the third text error information to the IE plugin;

the display module is further configured to display the third text error message.

9. The system of claim 7, further comprising:

the second routing inspection module is positioned on the shared platform server and used for detecting the communication state of the IE plug-in at preset second time intervals and detecting whether the second video stream is received or not; if the communication state is a disconnected state, generating a fourth error code; if the second video stream is not received, generating a fifth error code; and sending the fourth error code and/or the fifth error code to the Mserver;

the error code forwarding module is further configured to convert the fourth error code into fourth text error information conforming to a sip protocol when the fourth error code and/or the fifth error code are received, and/or convert the fifth error code into fifth text error information conforming to the sip protocol; and sending the fourth text error information and/or the fifth text error information to the IE plugin;

the display module is further configured to display the fourth text error information and/or the fifth text error information.

10. The system of claim 7, wherein the video stream signaling module further comprises:

the display module is further configured to display the stream adjustment initiation information.