CN116320309A - Monitoring method and monitoring equipment of inspection equipment - Google Patents

Abstract

The application provides a monitoring method of inspection equipment, which comprises the following steps: acquiring network camera information, wherein the network camera information comprises open network video interface forum ONVIF protocol information; determining target inspection equipment according to the network camera information; sending first information to target inspection equipment, wherein the first information is used for indicating the target inspection equipment to send an audio and video stream data packet to monitoring equipment; receiving a first audio and video stream data packet sent by target inspection equipment according to the indication of the first information, and analyzing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment; the original audio-video data is displayed in a monitor window. According to the method, the patrol equipment which can be connected with the same monitoring equipment is screened out through ONVIF protocol information, so that the monitoring equipment can play monitoring videos of the patrol equipment of different manufacturers, and unified monitoring is realized.

Description

Monitoring method and monitoring equipment of inspection equipment

Technical Field

The application belongs to the technical field of monitoring of inspection equipment, and particularly relates to a monitoring method and monitoring equipment of inspection equipment.

Background

With the development of monitoring technology, mobile monitoring systems are gradually rising, and the problems of limited monitoring angles and the like of fixed monitoring systems are solved. The inspection equipment is used as mobile monitoring equipment, can effectively manage the scene of mobile personnel, and has important security and protection effects in the fields of electric power, petrifaction, fire protection, railway traffic and the like.

However, at present, most of inspection devices of different manufacturers are based on respective internal standards, and often only monitoring devices provided by the manufacturers can play monitoring videos, so that the monitoring devices cannot be connected to the same system for unified monitoring, and universality is not high. When in actual use, because protocols defined inside the patrol equipment manufacturer are different, the compatibility problem of the patrol equipment is often required to be considered when the monitoring equipment is accessed, so that the developed technology has high cost and high application difficulty.

Therefore, how to enable the inspection equipment of different manufacturers to access the same monitoring equipment to realize unified monitoring is a technical problem to be solved urgently.

Disclosure of Invention

The embodiment of the application provides a monitoring method and monitoring equipment for inspection equipment, which can enable the inspection equipment of different manufacturers to be connected with the same monitoring equipment so as to realize unified monitoring.

In a first aspect, an embodiment of the present application provides a monitoring method of a patrol equipment, which is applicable to the monitoring equipment, and the method includes: acquiring network camera information, wherein the network camera information comprises Open Network Video Interface Forum (ONVIF) protocol information; determining target inspection equipment according to the network camera information; sending first information to target inspection equipment, wherein the first information is used for indicating the target inspection equipment to send an audio and video stream data packet to monitoring equipment; receiving a first audio and video stream data packet sent by target inspection equipment according to the indication of the first information, and analyzing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment; the original audio-video data is displayed in a monitor window.

In a possible implementation manner of the first aspect, when receiving a first audio/video stream data packet sent by the target inspection device according to the indication of the first information, and analyzing the first audio/video stream data packet, obtaining original audio/video data recorded by the target inspection device may include: establishing a real-time transport protocol (RTP) connection and a real-time transport control protocol (RTCP) connection with a target inspection device; periodically receiving a first audio and video stream data packet sent by target inspection equipment; and analyzing the first audio and video stream data packet to obtain the original audio and video data recorded by the target inspection equipment.

In a possible implementation manner of the first aspect, the method further includes: and sending second information to the target inspection equipment, wherein the second information comprises a response RTP packet and a response RTCP packet, and the second information is used for indicating the target inspection equipment to continuously send the first audio and video stream data packet to the monitoring equipment.

In a possible implementation manner of the first aspect, parsing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection device may include: when the data volume buffered in the receiving buffer area of the first audio/video stream data packet reaches the upper limit of capacity, a plurality of sub data packets contained in the first audio/video stream data packet are packed, so that audio/video data to be analyzed is obtained; and decoding the audio and video data to be analyzed through a decoder to obtain the original audio and video data recorded by the target inspection equipment.

In a possible implementation manner of the first aspect, the method further includes: the receive buffer is initialized prior to periodically receiving the first audio video stream data packet.

In a possible implementation manner of the first aspect, the method further includes: after sending the audio-video data to be parsed to the decoder, the receiving buffer is emptied.

In a possible implementation manner of the first aspect, when receiving the first av stream packet fails, the method further includes: prompting error information of failure in receiving in a monitoring window, wherein the error information is used for indicating to respond to a result of failure in receiving; and sending third information to the target inspection equipment, wherein the third information is used for indicating the target inspection equipment to stop sending the audio and video stream data packet to the monitoring equipment.

In a second aspect, an embodiment of the present application provides a monitoring method of an inspection device, which may be applied to the inspection device, where the method includes: receiving first information sent by monitoring equipment, wherein the first information comprises an access request of a target network camera, and the target network camera meets the Open Network Video Interface Forum (ONVIF) protocol standard; playing the target network camera according to the access request to obtain original audio and video data; encoding and unpacking original audio and video data to obtain a first audio and video stream data packet; and sending the first audio and video stream data packet to the monitoring equipment.

In a possible implementation manner of the second aspect, before sending the first av stream data packet to the monitoring device, the method further includes establishing an RTP connection and an RTCP connection with the monitoring device.

In a possible implementation manner of the second aspect, if the second information sent by the monitoring device is received, the first audio/video stream data packet is continuously sent to the monitoring device.

In a possible implementation manner of the second aspect, if the third information sent by the monitoring device is received, sending of the first audio/video stream data packet to the monitoring device is stopped.

In a third aspect, embodiments of the present application provide a monitoring device having a function of implementing the method in the first aspect or any possible implementation manner thereof. In particular, the apparatus comprises means for implementing the method of the first aspect or any possible implementation thereof.

In one embodiment thereof, the apparatus comprises:

an acquisition unit configured to acquire network camera information, where the network camera information includes Open Network Video Interface Forum (ONVIF) protocol information; determining target inspection equipment according to the network camera information;

the processing unit is used for sending first information to the target inspection equipment, wherein the first information is used for indicating the target inspection equipment to send an audio/video stream data packet to the monitoring equipment; receiving a first audio and video stream data packet sent by target inspection equipment according to the indication of the first information, and analyzing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment;

And the display unit is used for displaying the original audio and video data on the monitoring window.

In a fourth aspect, embodiments of the present application provide a patrol apparatus having a function of implementing the method in the second aspect or any possible implementation manner thereof. In particular, the device comprises means for implementing the method of the second aspect or any possible implementation thereof.

In one embodiment thereof, the apparatus comprises:

the receiving unit is used for receiving first information sent by the monitoring equipment, wherein the first information comprises an access request of a target network camera, and the target network camera meets the Open Network Video Interface Forum (ONVIF) protocol standard;

the processing unit is used for playing the target network camera according to the access request to obtain the original audio and video data; encoding and unpacking original audio and video data to obtain a first audio and video stream data packet;

and the sending unit is used for sending the first audio and video stream data packet to the monitoring equipment.

In a fourth aspect, an embodiment of the present application provides a monitoring system of a patrol equipment, where the monitoring system includes at least one patrol equipment and a monitoring device; the monitoring equipment is used for monitoring at least one inspection equipment; the monitoring device is a monitoring device capable of executing the method according to any implementation manner of the first aspect, and the at least one inspection device is an inspection device capable of executing the method according to any implementation manner of the second aspect.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium. The computer readable storage medium stores a computer program which when executed by a processor implements the method of any one of the implementations of the first or second aspect described above.

In a sixth aspect, embodiments of the present application provide a computer program product, which when run on a terminal device, causes the terminal device to perform the method of any one of the implementations of the first or second aspects described above.

Compared with the prior art, the embodiment of the application has the beneficial effects that: according to the monitoring method of the inspection equipment, the target inspection equipment meeting the access condition is screened out according to whether the network camera information contains the ONVIF protocol information of the open network video interface forum; after the monitoring equipment and the target inspection equipment are in communication connection, the audio and video data packet sent by the target inspection equipment is received and analyzed, and a monitoring picture is displayed on a monitoring window. According to the method, the correlation between the inspection equipment and the monitoring equipment is established through ONVIF protocol information, so that the monitoring equipment can be compatible with the inspection equipment of different manufacturers, that is, the audio and video recorded by the inspection equipment of different manufacturers can be played through the unified monitoring equipment without being played through the monitoring equipment special for each manufacturer, and therefore unified monitoring of the inspection equipment is achieved, and the method has universality and universality.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly introduce the drawings that are needed in the embodiments or the description of the prior art, it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an application scenario schematic diagram of a monitoring method of a patrol equipment according to an embodiment of the present application;

FIG. 2 is a network interaction diagram of the open network video interface Forum ONVIF protocol;

fig. 3 is a flow chart of a monitoring method of a patrol equipment according to an embodiment of the present application;

fig. 4 is a flow chart of a monitoring method of a patrol equipment according to another embodiment of the present application;

fig. 5 is a timing chart of a monitoring method of a patrol equipment according to another embodiment of the present application;

fig. 6 is a schematic block diagram of a monitoring method of a patrol equipment according to another embodiment of the present application;

fig. 7 is a schematic structural diagram of a monitoring device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a patrol equipment according to an embodiment of the present application;

Fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

Fig. 1 is an application scenario schematic diagram of a monitoring method of a patrol equipment according to an embodiment of the present application. The inspection device 110 and the monitoring device 120 are included in the figure, wherein the inspection device 110 communicates with the monitoring device 120 through a network.

As shown in fig. 1, in a mobile monitoring scenario, the inspection device 110 includes, but is not limited to, a handheld device, an intelligent robot, an inspection trolley, an unmanned aerial vehicle, and the like, and transmits "seen" content to the remote monitoring device 120 through a carried webcam, so that a monitoring person can observe the transmission screen. The monitoring devices include, but are not limited to, a video wall, a group of monitoring client terminals, and the like. The types of the inspection device 110 and the monitoring device 120 are not limited in the embodiments of the present application.

In the traditional scheme, the inspection equipment of different manufacturers often needs to be provided with monitoring equipment special for the manufacturers to realize operations such as audio and video playing, and the like, because the inspection equipment of different manufacturers processes recorded original audio and video in different modes, and the transmitted data formats are also different. After the monitoring equipment receives the transmitted content, the content needs to be further processed, and the processing mode corresponds to the mode of processing the original audio and video by the target inspection equipment. Therefore, in the traditional scheme, the inspection equipment and the monitoring equipment of different manufacturers cannot be paired to realize the monitoring function, and the system has no universality. For example, inspection device a is produced by vendor a and inspection device B is produced by vendor B. An enterprise originally uses the inspection equipment A and purchases the inspection equipment B later, and the inspection equipment A and the inspection equipment B cannot be uniformly connected to the same monitoring equipment for playing because of different processing modes of original audios and videos. And if the monitoring equipment which is produced by the purchasing manufacturer B and paired with the inspection equipment B is needed by a certain enterprise, the monitoring video cannot be played. According to the embodiment of the application, the target inspection equipment which can be connected with the same monitoring equipment is screened out according to whether the network camera information contains the open network video interface forum (Open Network Video Interface Forum, ONVIF) protocol information, after the monitoring equipment and the target inspection equipment are in communication connection, an audio and video data packet sent by the target inspection equipment is received and analyzed, a monitoring picture is displayed on a monitoring window, and audio and video recorded by the inspection equipment of different manufacturers can be played through unified monitoring equipment, so that the application has universality.

Fig. 2 is a network interaction diagram of the open network video interface forum ONVIF protocol.

The ONVIF is a global open industry forum that creates a standard for how internet protocol (Internet Protocol, IP) products in the video surveillance and other physical security fields communicate with each other, focusing on the interface communication between clients and servers. With the networked application of video monitoring, the division of industry chain is finer and finer. Some manufacturers are dedicated to cameras, some manufacturers are dedicated to video servers, some manufacturers are dedicated to platforms, etc., and then integrated by an integrator for providing to end users. And proprietary protocols are not adequate for this industry collaboration mode. The ONVIF protocol allows video products from different vendors to be used in the same system and in the same language. And as long as each manufacturer formulates a product according to ONVIF protocol specifications, the intercommunication and interconnection of different products of different manufacturers can be realized. Meanwhile, the protocol is more standard and perfect, and meets the requirements of almost all video monitoring systems in the current market.

The ONVIF protocol specification introduces the Web Service concept into the video monitoring field, where interfaces defined by the device management and control section are provided in the form of Web services. Each terminal device supporting the ONVIF specification has to provide Web Service corresponding to the function. The data interaction between the server and the client uses standard simple object access protocol (Simple Object Access Protocol, SOAP). Other parts of the ONVIF, such as audio and video streaming, are performed using established multimedia transport protocols, such as Real-time transport protocol (Real-time Transport Protocol, RTP)/Real-time streaming protocol (Real Time Streaming Protocol, RTSP).

Taking an example of an Internet Protocol CAMERA (IPC), the actual functions of the IPC are abstracted into Web service services, and a control unit of the video monitoring system appears in the form of a client and completes the actual control operation in the form of a Web request. That is, IPC is a Service end of Web Service, and Web Service interfaces provided by the IPC need to conform to the ONVIF protocol specification, and these interfaces are in a network Service description language (Web Services Description Language, WSDL) document specified by the ONVIF, and clients communicate with IPC through these ONVIF specification interfaces, for example: basic information (manufacturer information, version information and the like) of the IPC is acquired, the system date and time of the IPC are modified, the network configuration (IP address, subnet mask and the like) of the IPC is modified, and parameters of the IPC camera such as video resolution, code rate, frame rate and the like are acquired or modified.

The network interaction procedure of the ONVIF protocol is shown in fig. 2. In fig. 2, the service side includes a service provider 210, a service provider 211, and a service provider 212, and the client includes a service requester 220, a service requester 221, and a service requester 222. In this embodiment of the application, the inspection device belongs to a service provider, and the monitoring device belongs to a service requester. The service provider designs to implement Web service, issues the service through Web service mediator after debugging is correct, and provides WSDL document 230; the service requester requests a specific service from the Web service mediator; the Web service mediator returns Web service description information meeting the conditions, namely a WSDL document, to the service requester, and the document can be read by various machines supporting Web services; and generating corresponding SOAP messages by using the description information returned from the Web service mediator, and sending the SOAP messages to the service provider to realize the call of the Web service.

Fig. 3 is a flowchart of a monitoring method of a patrol equipment according to an embodiment of the present application.

S301, the monitoring equipment acquires the information of the network camera.

The network camera information may include ONVIF protocol information, among others.

The network camera combines the traditional camera and the network video technology, and can digitize the video signals transmitted by the camera, compress the video signals by the high-efficiency compression chip and transmit the video signals to the Web server through the network bus. The network camera can be combined with a router, a switch, a network hard disk video recorder (Net Video Recorder, NVR) and other devices to form a monitoring system.

Accessing a network camera, calling a Web service interface provided by the network camera, and firstly acquiring information of the network camera. In the conventional Web Service call mode, a client knows in advance an address of a target Service, such as an IP address or a domain name, at design time, and makes a Service call based on the address. When the client does not know the address of the target service in advance, the client is required to actively search and acquire the camera information in the network, and the information comprises ONVIF protocol information. The process of obtaining the above information is also referred to as device discovery.

In one possible implementation, according to the ONVIF protocol specification, the monitoring device (client) uses the user datagram protocol (User Datagram Protocol, UDP) to send a probe message to a certain multicast address fixed in the network segment, and the network camera listens on the fixed port, after receiving the probe message, sends a reply message to the monitoring device, the reply message including the ONVIF protocol information, such as the IP address of the camera itself, the universally unique identification code (Universally Unique Identifier, UUID), and the service address. The monitoring device can select the camera to be played according to the IP address and request the service from the corresponding service address. In one example, the multicast address of the probe message sent by the monitoring device is 239.255.255.250:3702, the service address of the received camera is http://192.168.0.64:8099/onvif/device_service, and the monitoring device can request the service provided by the camera by accessing the address.

In another possible implementation manner, the frame code can be quickly generated by using the gSOAP tool according to the remotediscovery.wsdl document of the ONVIF standard, and the generated function interface is directly called to search the network camera so as to acquire ONVIF protocol information of the network camera.

S302, the monitoring equipment determines target inspection equipment according to the network camera information.

The monitoring equipment screens out the network cameras carried on the inspection equipment according to the acquired network camera information, and determines the inspection equipment where the screened network cameras are located as target inspection equipment.

The target inspection equipment can be mobile monitoring equipment, and comprises a handheld inspection terminal, an inspection robot and the like, and is used for realizing mobile monitoring on complex sites such as lines, pipelines and equipment.

The network cameras searched by the monitoring equipment are not carried by the inspection equipment, but are fixed-position cameras. It is necessary to delete the corresponding information in the searched network camera list. In addition, there are also cases where a network camera is mounted on the patrol equipment but not searched because the network camera does not meet the ONVIF protocol standard, cannot be found by the monitoring equipment, and the patrol equipment where the network camera is located is ignored by the monitoring equipment and is not listed as the target patrol equipment.

S303, the monitoring equipment sends first information to the target patrol equipment.

The first information is used for indicating the target inspection equipment to send the audio and video stream data packet to the monitoring equipment. The first information includes a network camera access request. And the target inspection equipment plays the network camera with the corresponding address according to the request and sends an audio and video stream data packet to the monitoring equipment.

S304, the target inspection equipment plays the target network camera according to the access request to obtain the original audio and video data.

The target inspection equipment plays the target network camera, the target camera records the optical information of the scene seen by the target network camera as internal electric information, and the original uncompressed video stream, namely the original audio and video stream, is output. The format of the original audio and video can be YUV format, rawRGB format or MJPG/JPEG format. Preferably, the original audio and video uses YUV format as output format, where "Y" represents brightness (luminence or Luma), that is, gray scale value, and "U" and "V" represent chromaticity (Chroma) to describe image color and saturation for specifying the color of the pixel.

S305, the target inspection device encodes and unpacks the original audio and video data to obtain a first audio and video stream data packet.

The pictures seen by the camera often need to be transmitted to a designated place in a long distance, and the quantity of original audio and video data transmitted by a playing link is large, so that the transmission is inconvenient. Therefore, the original audio and video needs to be compressed before transmission, and this compression process is also called coding.

In one possible implementation, the video encoding method may use an MPEG series encoding method including MPEG1, MPEG2, MPEG4 AVC, etc.

In another possible implementation, the video encoding method may also use the h.26x series, including h.261, h.263, h.264, and so on.

In one possible implementation, the audio coding method may use AAC or MP3 coding.

After encoding, the data volume of the original audio and video is greatly compressed. In order to ensure the accuracy and Real-time performance of transmission, the encoded original audio and video are disassembled and packaged through a Real-time transmission protocol (Real-time Transport Protocol, RTP) to obtain a plurality of sub-data packets. These sub-packets, i.e. the first av stream packets, comprise two major parts: the RTP protocol provides information and data payload. The information provided by the RTP protocol includes: time stamp (for synchronization), sequence number (for packet loss and reordering detection), and payload format (for encoding format of the description data). The data load comprises the data fragments obtained by slicing the coded original audio and video according to the maximum length of the RTP packet.

S306, the target inspection device sends a first audio/video stream data packet to the monitoring device.

And the original audio and video data is subjected to the encoding and unpacking to obtain a first audio and video stream data packet. The target inspection device can send the first audio/video stream data packet to the monitoring device through a UDP protocol format.

S307, the monitoring equipment receives a first audio and video stream data packet sent by the target inspection equipment according to the indication of the first information, and analyzes the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment.

Because the first audio and video stream data packet is obtained after the target inspection device is encoded and unpacked, the monitoring device cannot directly play the first audio and video stream data packet after receiving the first audio and video stream data packet, and needs to analyze the data packet to obtain the original audio and video data, and then display the original audio and video data in the monitoring window.

S308, the monitoring device displays the original audio and video data on the monitoring window.

The monitoring equipment obtains original audio and video through analysis, wherein the original audio and video refers to uncoded audio and video recorded by the target inspection equipment. And the monitoring equipment displays the original audio and video on a monitoring window after obtaining the original audio and video, so that the playing of the audio and video is realized.

According to the monitoring method of the inspection equipment, the target inspection equipment meeting the access condition is determined according to whether the network camera information contains the ONVIF protocol information of the open network video interface forum; after the monitoring equipment and the target inspection equipment are in communication connection, the audio and video data packet sent by the target inspection equipment is received and analyzed, and a monitoring picture is displayed on a monitoring window. The method establishes the association between the inspection equipment and the monitoring equipment through ONVIF protocol information, so that the monitoring equipment can be compatible with the inspection equipment of different manufacturers. In the method, the audio and video recorded by the inspection equipment of different manufacturers can be played through the unified monitoring equipment without depending on the monitoring equipment special for each manufacturer. For example, in the application scenario shown in fig. 1, the patrol equipment a and the patrol equipment B both meet the ONVIF protocol standard, and an enterprise can access the patrol equipment a and the patrol equipment B into the monitoring equipment C by the method, so that the monitoring equipment can uniformly monitor the patrol equipment, and the method has universality and universality. And a certain enterprise does not need to purchase monitoring equipment special for each of the manufacturer a and the manufacturer b, so that the enterprise cost is saved.

Fig. 4 is a flowchart of a monitoring method of a patrol equipment according to another embodiment of the present application. Fig. 4 can be regarded as a specific example of the method shown in fig. 3.

The contents of S401 to S405 in fig. 4 are the same as those of S301 to S305 in the embodiment of fig. 3, and reference may be made to the description of S301 to S305 for brevity. The steps other than S401 to S405 of fig. 4 are described below.

S406, the monitoring equipment establishes RTP connection and RTCP connection with the target patrol equipment.

The data transmission protocol RTP is used for transmitting data in real time. Control protocol RTCP for quality of service (Quality of Service, qoS) feedback and synchronizing media streams.

In one possible implementation, the monitoring device establishes a session with the target patrol device, the RTP uses an even port number to receive the transmission data, and the corresponding RTCP uses the next adjacent odd port number. For example, if the port number selected by RTP is 5004, then RTCP belonging to the same session selects an odd port number added to 1, i.e., 5005. It should be understood that the numerical values in this example are only one example and are not limiting.

The monitoring equipment establishes RTP connection and RTCP connection with the target inspection equipment, provides time information and stream synchronization through the RTP connection, and realizes quality service through the RTCP connection, and the RTP connection and the RTCP connection are matched with each other to realize real-time transmission of the monitoring audio/video stream.

S407, the target inspection device sends the first audio and video stream data packet.

After the monitoring equipment establishes RTP and RTCP connection with the target inspection equipment, the target inspection equipment starts to send audio and video stream sub-data packets encapsulated according to RTP protocol, namely first audio and video stream data packets, to the monitoring equipment. In addition, in the sending process, the target patrol equipment monitors whether ports corresponding to RTP and RTCP receive response data packets sent by the monitoring equipment or close the connection request.

S408, the monitoring device initializes the receiving buffer.

The monitoring device initializes a receive buffer prior to receiving the audio-video stream packets. And caching the received audio and video data packets in a receiving buffer area, reordering and grouping the disordered packets according to the encapsulation information (such as the packet sequence number, the time stamp and the like) of the data packets, and sending the reordered packets to a decoder for decoding. Because the data load in the audio and video stream data packet is only a data fragment, and network transmission is inevitably delayed, the condition of disorder and packet loss often exists when the monitoring equipment receives the data packet, and the influence of disorder and packet loss on analysis of audio and video can be reduced by setting the receiving buffer zone.

S409, the monitoring device periodically receives the first audio/video stream data packet.

And the monitoring equipment receives the first audio and video stream data packet sent by the target inspection equipment in each session period.

In one implementation, the monitoring device continuously receives the first audio and video stream data packet, and after receiving the data packet in a session period, sends a response RTP packet and a response RTCP packet to the target inspection device, that is, second information, where the second information is used to instruct the target inspection device to continuously send the first video stream data packet, and perform the next session.

In one implementation, when an error occurs in the received data packet, the monitoring device stops receiving and sends error information, that is, third information, to the target inspection device, which is used to instruct the target inspection device to stop sending the audio and video data packet and interrupt the current session.

The received audio and video stream data packet in one session period can be processed to display the original audio and video data and play the monitoring video clip. And then receiving and processing the data packet of the next session period, and playing the data packet, so that the whole audio and video data analysis is not required to be completed, the delay of the downloading time is avoided, and the timeliness of monitoring is ensured.

S410, grouping a plurality of sub-data packets in the first audio/video stream data packet to obtain audio/video data to be analyzed.

And when the data volume of the receiving buffer zone reaches the upper limit of capacity, grouping a plurality of sub-data packets in the first audio/video stream data packet in the buffer zone, and analyzing the load data in the sub-data packets according to RTP and RTCP protocols to obtain the audio/video stream data which is not decoded, namely the audio/video data to be analyzed.

S411, a decoder of the monitoring device decodes the audio and video data to be analyzed to obtain the original audio and video data.

According to the load format provided by the RTP protocol in step S305, the decoder decodes the audio/video data to be analyzed to obtain the original audio/video data recorded by the target inspection device.

S412, the monitoring device empties the receiving buffer.

In one implementation, after decoding by the decoder, the monitoring device empties the receiving buffer area to reserve space for receiving the audio/video stream data packet of the next session period. Meanwhile, after decoding is finished, the buffer area is emptied, the data in the buffer area is received as backup, the data packet loss caused by decoding failure can be effectively prevented, and the data transmission is more accurate.

In another implementation, the monitoring device empties the receive buffer after sending the packetized data to the decoder, that is, S412 is performed immediately after S410. Therefore, the problem of buffer overflow caused by the fact that the receiving buffer is still receiving the data packet during the decoding period of the decoder can be effectively avoided, and the data transmission is safer. It should be understood that the monitoring device only needs to empty the receiving buffer before the next session starts to receive the audio/video data packet.

S413, the monitoring device displays the original audio and video data on the monitoring window.

This step can be regarded as a specific example of step S308 in the embodiment of fig. 3.

The received audio and video stream data packet in one session period can be processed to display the original audio and video data and play the monitoring video clip. And then receiving and processing the data packet of the next session period, and playing the data packet, so that the whole audio and video data analysis is not required to be completed, the delay of the downloading time is avoided, and the timeliness of monitoring is ensured.

And S414, the monitoring equipment sends second information to the target inspection equipment when the first audio and video stream data packet is successfully received.

The monitoring device receives the first audio and video stream data packet successfully, sends a response RTP packet and a response RTCP packet to the target inspection device, namely second information, and is used for indicating the target inspection device to continue sending the first video stream data packet and carrying out the next session.

S415, when the monitoring equipment fails to receive the first audio/video stream data packet, the monitoring window prompts error information of the failure to receive.

The monitoring device prompts error information of the reception failure in the monitoring window and is used for indicating to respond to the result of the reception failure. For example, the monitor window pops up a prompt box "data loading failed, please retry", and the monitor personnel can reenter the monitor window according to the prompt message.

S416, the monitoring equipment sends third information to the target inspection equipment when the first audio and video stream data packet fails to be received.

The monitoring device sends error information, namely third information, to the target inspection device, and the error information is used for indicating the target inspection device to stop sending the audio and video data packet and interrupting the current session.

Fig. 5 is a timing chart of a monitoring method of a patrol equipment according to another embodiment of the present application.

As shown in fig. 5, when the monitoring device performs video monitoring, a UI thread is used to display a front-end interactive interface, an IPC device thread is used to perform device management and control, and a data receiving thread is used to receive and analyze an audio/video stream. Specifically, when the monitoring device plays the audio and video stream transmitted by the inspection device, the method comprises the following steps.

S501, a monitoring window acquires a camera.

This step can be regarded as a specific example of S401.

The monitoring window acquires camera information, wherein the information comprises ONVIF protocol information, and a service address and the like of the camera can be obtained to call Web service provided by the camera, so as to determine target inspection equipment which can be accessed to the monitoring equipment.

S502, the UI thread sends an IPC opening request to the IPC device thread.

After the UI thread obtains the information of the network camera, the IPC equipment thread is called, and opening request information MSG_IPC_OPEN_REQ is sent to the target inspection equipment through the IPC equipment thread.

S503, the UI thread pops up the requesting window.

S504, the IPC device thread calls to open the camera service and plays the camera.

S502 and S504 can be regarded as one specific example of S403.

After receiving the opening request of the UI thread, the IPC equipment thread calls the service address to open the camera service, sends playing request information MSG_IPC_PLAY_REQ to the address, and PLAYs the camera.

S505, the IPC device thread creates a data receiving thread, and if the creation is successful, the steps S506 to S513 are executed; if the creation fails, steps S514 to S515 are performed.

S506, in the data receiving thread, the monitoring equipment establishes RTP and RTCP connection with the target patrol equipment. This step may correspond to step S406 of the embodiment in fig. 4.

S507, initializing the video area and the reception buffer.

This step may correspond to step S408 of the embodiment of fig. 4.

Since the received audio and video stream data packets also need to be subjected to subsequent packetizing and decoding operations, the data receiving thread needs to make room to store the received audio and video stream data packets which are not packetized and decoded yet, and therefore, before receiving the data packets, the video area and the receiving buffer area need to be initialized first.

S508, the data packet is circularly received.

This step can be regarded as a specific example of S409.

In a session period, the received audio and video stream data packet is processed to play the original audio and video clip. And then receiving and processing the data packet of the next session period, so that the delay of the completion of the analysis and the downloading time of the whole audio and video data is not required to be waited, and the timeliness of monitoring can be ensured.

S509, multi-packet grouping.

This step can be regarded as a specific example of S410.

And carrying out multi-packet grouping on the data packet circularly received in the previous step to obtain the audio and video data to be analyzed.

S510, decoding by a decoder to obtain original YUV data.

This step can be regarded as a specific example of S411.

And inputting the audio and video data to be analyzed into a decoder of the monitoring equipment for decoding, so that the audio and video data in the original YUV format recorded by the target inspection equipment can be obtained.

S511, the response RTP packet and the response RTCP packet are transmitted to the target patrol apparatus, and then S512 is executed.

This step can be regarded as a specific example of S413.

After the monitoring device successfully receives and analyzes the original audio and video data, the monitoring device sends a notification of updating the real-time picture to the UI thread, and sends a response RTP packet and a response RTCP packet to the target inspection device, namely the second information in S413, wherein the second information is used for indicating the target inspection device to continue sending the audio and video data to the monitoring device.

S512, the IPC device thread sends a notification of updating the real-time screen to the UI thread.

S513, the UI thread closes the request window and displays the original audio and video data in real time.

This step corresponds to step S413 of the above embodiment.

And then re-executing S508 to obtain the audio/video stream data of the next session period.

S514, the IPC device thread sends a notification of failure in reception to the UI thread and sends a request for closing the network camera to the service address, and the method comprises the following steps:

sending request information MSG_IPC_CLOSE_REQ for closing the network camera to the service address;

stopping the data receiving thread;

closing the IPC equipment;

closing RTP and RTCP connections;

releasing the video initialization area.

S515, the UI thread closes the request window and pops up a prompt box for retry or error information.

Fig. 6 is a schematic block diagram of a monitoring method of a patrol equipment according to another embodiment of the present application.

As shown in fig. 6, the patrol equipment 610 accesses the ONVIF protocol module 620 to implement mobile monitoring. The ONVIF protocol module 620 includes a Web server module 621 and an RTSP module 622. The Web server module 621 is used for implementing management and control of devices, and the RTSP module 622 is used for implementing management and control of audio/video streams.

In one possible implementation, the network camera information of the inspection device in the above embodiment may be provided by the Web server module 621.

The processing and transmission of the recorded audio-video stream in one possible implementation is performed by RTSP module 622. In one example, RTSP module 622 may implement S304 and S305 in the above embodiments. In another example, RTSP module 622 may implement S404 and S405 in the above-described embodiments.

The foregoing description mainly refers to the accompanying drawings for a monitoring method of a patrol equipment according to an embodiment of the present application. It should be understood that, although the steps in the flowcharts related to the embodiments described above are shown in order, these steps are not necessarily performed in the order shown in the figures. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

A monitoring device according to an embodiment of the present application is described below with reference to the accompanying drawings. For brevity, the description of the monitoring device will be omitted appropriately, and the relevant content may refer to the relevant description in the monitoring method of the inspection device, and the description will not be repeated.

Fig. 7 is a schematic structural diagram of a monitoring device according to an embodiment of the present application.

As shown in fig. 7, the apparatus 1000 includes an acquisition unit 1001, a processing unit 1002, and a display unit 1003.

In one possible implementation, the obtaining unit 1001 may be configured to obtain network camera information, where the network camera information may include ONVIF protocol information. Based on the network camera information, the acquisition unit 1001 may determine the target patrol apparatus. The processing unit 1002 may be configured to send first information to the target inspection device, where the first information is used to instruct the target inspection device to send an audio/video stream data packet to the monitoring device. The processing unit 1002 receives a first audio/video stream data packet sent by the target inspection device according to the indication of the first information, and parses the first audio/video stream data packet to obtain original audio/video data recorded by the target inspection device. The display unit 1003 may be used to display the original audio-video data on a monitor window.

In one embodiment, the processing unit 1002 is specifically configured to establish an RTP connection and an RTCP connection with the target patrol device; periodically receiving a first audio and video stream data packet sent by target inspection equipment; and analyzing the first audio and video stream data packet to obtain the original audio and video data recorded by the target inspection equipment.

In one embodiment, the processing unit 1002 is further configured to send second information to the target patrol device, where the second information includes a response RTP packet and a response RTCP packet, and is configured to instruct the target patrol device to continue sending the first av stream data packet to the monitoring device.

In an embodiment, the processing unit 1002 is specifically further configured to group a plurality of sub-packets included in the first audio/video stream data packet when the amount of data buffered in the receiving buffer of the first audio/video stream data packet reaches an upper limit of capacity, so as to obtain audio/video data to be parsed; and decoding the audio and video data to be analyzed through a decoder to obtain the original audio and video data recorded by the target inspection equipment.

In one embodiment, the processing unit 1002 is further configured to initialize the receive buffer before periodically receiving the first av stream data packet.

In one embodiment, the processing unit 1002 is further configured to empty the receiving buffer after sending the audio-video data to be parsed to the decoder.

In one embodiment, when the first av stream packet fails to be received, the display unit 1003 is further configured to prompt, in the monitoring window, an error message of the reception failure, and to indicate that the response to the reception failure result is performed; the processing unit 1002 is further configured to send third information to the target inspection device, where the third information is used to instruct the target inspection device to stop sending the audio/video stream data packet to the monitoring device.

In a possible implementation manner, the apparatus 1000 may further include a storage unit, configured to store data such as network camera information, an audio/video stream packet, and the like. The memory unit may be integrated in any one of the above units, or may be a unit independent of all the above units.

Fig. 8 is a schematic structural diagram of a patrol equipment according to an embodiment of the present application.

As shown in fig. 8, the apparatus 2000 includes a receiving unit 2001, a processing unit 2002, and a transmitting unit 2003.

In one possible implementation manner, the receiving unit 2001 is configured to receive first information sent by the monitoring device, where the first information includes an access request of a target network camera, where the target network camera meets an open network video interface forum ONVIF protocol standard; the processing unit 2002 is used for playing the target network camera according to the access request to obtain original audio and video data; encoding and unpacking original audio and video data to obtain a first audio and video stream data packet; the sending unit is used for sending the first audio and video stream data packet to the monitoring equipment.

In one embodiment, the processing unit 2002 is specifically further configured to establish an RTP connection and an RTCP connection with the monitoring device.

In one embodiment, the processing unit 2002 is further configured to, if receiving the second information sent by the monitoring device, continue to send the first av data packet to the monitoring device.

In one embodiment, the processing unit 2002 is further configured to stop sending the first av data packet to the monitoring device if the third information sent by the monitoring device is received.

In a possible implementation manner, the apparatus 2000 may further include a storage unit, configured to store data such as network camera information, an audio/video stream packet, and the like. The memory unit may be integrated in any one of the above units, or may be a unit independent of all the above units.

Fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 9, the computer device 3000 of this embodiment includes: at least one processor 3100 (only one shown in fig. 9), a memory 3200, and a computer program 3210 stored in the memory 3200 and executable on the at least one processor 3100, the steps in the above embodiments being implemented when the processor 3100 executes the computer program 3210.

The processor 3100 may be a central processing unit (Central Processing Unit, CPU), but the processor 3100 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

Memory 3200 may in some embodiments be an internal storage unit of computer device 3000, such as a hard disk or memory of computer device 3000. Memory 3200 may also be an external storage device of computer device 3000 in other embodiments, such as a plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card) or the like, which are provided on computer device 3000. Further, memory 3200 may also include both internal and external storage units of computer device 3000. The memory 3200 is used to store an operating system, application programs, boot Loader (Boot Loader) data, other programs, and the like, such as program codes of computer programs, and the like. The memory 3200 may also be used to temporarily store data that has been output or is to be output.

It should be noted that, because the content of information interaction and execution process between the above units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein again.

It will be apparent to those skilled in the art that the above-described functional units are merely illustrated in terms of division for convenience and brevity, and that in practical applications, the above-described functional units and modules may be allocated to different functional units or modules according to needs, i.e., the internal structure of the apparatus may be divided into different functional units or modules to perform all or part of the above-described functions. The functional units in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present application. The specific working process of the units in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

The embodiments of the present application also provide a computer readable storage medium storing a computer program, where the computer program when executed by a processor implements steps of the foregoing method embodiments.

Embodiments of the present application provide a computer program product enabling the implementation of the above-mentioned methods when the computer program product is run on a computer.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not limit the implementation process of the embodiment of the present application in any way. In the description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

Furthermore, in the description of the present application and the claims, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically noted.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus, computer device, and method may be implemented in other manners. For example, the apparatus, computer device embodiments described above are merely illustrative, e.g., the partitioning of elements is merely a logical functional partitioning, and there may be additional partitioning in actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims (10)

1. The monitoring method of the inspection equipment is applied to the monitoring equipment and is characterized by comprising the following steps of:

acquiring network camera information, wherein the network camera information comprises open network video interface forum ONVIF protocol information;

determining target inspection equipment according to the network camera information;

sending first information to the target inspection equipment, wherein the first information is used for indicating the target inspection equipment to send an audio/video stream data packet to the monitoring equipment;

receiving a first audio and video stream data packet sent by the target inspection equipment according to the indication of the first information, and analyzing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment;

And displaying the original audio and video data in a monitoring window.

2. The method of claim 1, wherein the receiving the first audio and video stream data packet sent by the target inspection device according to the indication of the first information, and analyzing the first audio and video stream data packet, to obtain the original audio and video data recorded by the target inspection device, includes:

establishing a real-time transmission protocol RTP connection and a real-time transmission control protocol RTCP connection with the target inspection equipment;

periodically receiving the first audio and video stream data packet sent by the target inspection equipment;

and analyzing the first audio and video stream data packet to obtain the original audio and video data recorded by the target inspection equipment.

3. The method of claim 2, wherein the method further comprises:

and sending second information to the target inspection equipment, wherein the second information comprises a response RTP packet and a response RTCP packet, and the second information is used for indicating the target inspection equipment to continue sending the first audio/video stream data packet to the monitoring equipment.

4. The method of claim 2, wherein the parsing the first av stream packet to obtain the original av data recorded by the target inspection device includes:

When the data volume buffered in the receiving buffer area of the first audio/video stream data packet reaches the upper limit of capacity, a plurality of sub data packets contained in the first audio/video stream data packet are packed, so that audio/video data to be analyzed is obtained;

and decoding the audio and video data to be analyzed through a decoder to obtain the original audio and video data recorded by the target inspection equipment.

5. The method of claim 4, wherein the method further comprises:

initializing the receiving buffer zone before periodically receiving the first audio/video stream data packet;

and after the audio and video data to be analyzed are sent to the decoder, the receiving buffer area is emptied.

6. The method of claims 1 to 5, wherein when receiving the first av stream packet fails, the method further comprises:

prompting error information of failure in receiving in a monitoring window, wherein the error information is used for indicating to respond to a result of failure in receiving;

and sending third information to the target inspection equipment, wherein the third information is used for indicating the target inspection equipment to stop sending the audio and video stream data packet to the monitoring platform.

7. The monitoring method of the inspection equipment is applied to the inspection equipment and is characterized by comprising the following steps of:

Receiving first information sent by monitoring equipment, wherein the first information comprises an access request of a target network camera, and the target network camera meets the ONVIF protocol standard of an open network video interface forum;

playing the target network camera according to the access request to obtain original audio and video data;

encoding and unpacking the original audio and video data to obtain a first audio and video stream data packet;

and sending the first audio and video stream data packet to the monitoring equipment.

8. A monitoring device, comprising:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring network camera information, and the network camera information comprises open network video interface forum ONVIF protocol information; determining target inspection equipment according to the network camera information;

the processing unit is used for sending first information to the target inspection equipment, wherein the first information is used for indicating the target inspection equipment to send an audio and video stream data packet to the monitoring equipment; receiving a first audio and video stream data packet sent by the target inspection equipment according to the indication of the first information, and analyzing the first audio and video stream data packet to obtain original audio and video data recorded by the target inspection equipment;

And the display unit is used for displaying the original audio and video data on a monitoring window.

9. A patrol apparatus, comprising:

the receiving unit is used for receiving first information sent by the monitoring equipment, wherein the first information comprises an access request of a target network camera, and the target network camera meets the ONVIF protocol standard of the open network video interface forum;

the processing unit is used for playing the target network camera according to the access request to obtain original audio and video data; encoding and unpacking the original audio and video data to obtain a first audio and video stream data packet;

and the sending unit is used for sending the first audio/video stream data packet to the monitoring equipment.

10. A monitoring system of a patrol equipment, characterized in that the monitoring system comprises at least one patrol equipment and a monitoring equipment; the monitoring equipment is used for monitoring the at least one inspection equipment; the at least one inspection device is an inspection device capable of performing the method of claim 7, and the monitoring device is a monitoring device capable of performing the method of any one of claims 1 to 6.

Images