CN110324568B

CN110324568B - Network video monitoring device

Info

Publication number: CN110324568B
Application number: CN201810272031.0A
Authority: CN
Inventors: 王明; 刘欢欢; 谢东亮; 付祺伟
Original assignee: Xilinx Technology Beijing Ltd
Current assignee: Xilinx Technology Beijing Ltd
Priority date: 2018-03-29
Filing date: 2018-03-29
Publication date: 2021-07-09
Anticipated expiration: 2038-03-29
Also published as: CN110324568A

Abstract

It is proposed a network video monitoring apparatus (200), comprising: a lens and sensor module (202), a video processing and transmission module (203), a power and interface module (204), and a Field Programmable Gate Array (FPGA) module (205). The FPGA module (205) is used for receiving the video data from the video processing and transmitting module (203), carrying out big data analysis processing on the received video data, and sending the result of the big data analysis processing back to the video processing and transmitting module (203). The video processing and transmission module (203) determines data to be transmitted over the network based on the results of the big data analysis process received from the FPGA module (205). Through the scheme, the upgrading from the traditional network camera to the artificial intelligent network camera can be realized quickly, and the security monitoring requirements under most scenes can be met.

Description

Network video monitoring device

Technical Field

The invention relates to data communication, in particular to a network video monitoring device.

Background

The network camera is taken as the most representative third-generation networked video monitoring product, integrates multiple functions of video and audio acquisition, intelligent coding compression, network transmission and the like, is widely accepted by the market in the field of industrial safety precaution, and more security protection items are implemented by network camera products.

The video resolution of the network camera is higher and higher, and higher requirements are put forward for network transmission bandwidth and network storage capacity. The large-scale application of the network camera generates massive video data, how to find a security object needing attention from the massive video, and great challenges are brought to the traditional manual retrieval mode. In order to solve these problems, the conventional solutions are: purchasing more network bandwidth; purchasing a larger capacity storage device; more manpower is used to analyze the video or the video stored by the network camera is analyzed in a centralized way by computer resources.

It is obvious that the above solution brings about the problems of time and labor consumption and money consumption.

Disclosure of Invention

The embodiment of the invention provides a network video monitoring device, wherein a security object is accurately and automatically found at the front end of a network camera, the behavior of the security object is recorded, only relevant valuable video data is reserved for returning, and useless video information with large specific gravity is not returned any more. Specifically, by adding an FPGA module in the traditional network camera, the video data in the camera is processed by the FPGA module and then determined which data and information are transmitted back through the network, so that the problems of time consumption, labor consumption and financial consumption in the prior art are solved.

To achieve the object of the present invention, according to a first aspect of the present invention, a network video monitoring apparatus is provided. The device includes: the camera lens and sensor module is used for acquiring an original video material; the video processing and transmitting module is used for processing the original video material and then transmitting the processed original video material through a network; and the power supply and interface module is used for supplying power to each module and providing a transmission interface. The network video monitoring device further comprises a Field Programmable Gate Array (FPGA) module which is used for receiving the video data from the video processing and transmitting module, carrying out big data analysis processing on the received video data and sending the result of the big data analysis processing back to the video processing and transmitting module. And the video processing and transmitting module determines data to be transmitted through the network according to the analysis and processing result of the big data received from the FPGA module.

By adopting the FPGA module, the video data which is just collected can be properly processed at the front end of the network video monitoring device, so that the problems of network transmission and large-capacity video storage are avoided.

Preferably, the network video monitoring device may further comprise a high-speed digital interface for transmitting video data from the video processing and transmitting module to the FPGA module.

Preferably, the high-speed digital interface is a BT1120 or RGMII interface.

The use of the high-speed digital interface, particularly the BT1120 or RGMII interface, enables the collected video data to be transmitted to the FPGA module as soon as possible, so that the analysis and processing of the video data can be completed in time at the front end of the network video monitoring device.

Preferably, the network video monitoring device may further include a common digital interface for transmitting a result of the big data analysis processing from the FPGA module to the video processing and transmitting module.

Preferably, the common digital interface is an SPI or UART interface.

For the result of big data analysis and processing, the capacity is completely inferior to that of video data, so that the ideal effect can be achieved even if a common digital interface is adopted. Therefore, the high-speed digital interface is saved and is used for transmitting the video data from the video processing and transmitting module to the FPGA module as much as possible.

Preferably, the big data analysis process may be an artificial intelligence process, and the result of the process may be structured information in the video data. More specifically, the artificial intelligence process may be an image recognition and tracking process, and the result of the process may be the identification of a tracked object in the video data.

The key technology of the intelligent security system is the identification of target objects, such as face identification, gesture identification and the identification of other specific target objects. In addition to recognition, tracking of the respective target is often required. Adopt artificial intelligence to handle in the FPGA module, more specifically adopt image recognition and tracking to handle, can obtain key structural information from video data to not only can discern and can also mark the tracking object in the video data.

Preferably, the network video monitoring device may further include a housing for enclosing the lens and sensor module, the video processing and transmission module, the power supply and interface module, and the FPGA module in the housing. The video processing and transmission module is used for dissipating heat by contacting with a boss which is upwards raised at the bottom in the shell and a heat conducting gasket, and the FPGA module is used for dissipating heat by contacting with the boss which is downwards raised at the top in the shell and the heat conducting gasket.

After the artificial intelligence work is added into the whole network video monitoring device, certain influence is brought to the power consumption and the heat dissipation of the whole device. In the invention, possible influence is suppressed by adopting the FPGA module with acceptable power consumption and adopting an additional heat dissipation measure. For example, in the original network video monitoring device, a heat conducting gasket is arranged on a boss which is arranged upwards at the bottom in the shell and is in contact with a video processing chip (a video processing and transmission module), so that the purpose of heat dissipation is achieved; after the FPGA module is added, the heat conducting gasket can be added through the downward boss on the top in the shell and is in contact with the FPGA module, and therefore the effect of extra heat dissipation is achieved.

Besides solving the defects of the prior art, the invention also has the following beneficial effects: firstly, the power consumption of the FPGA module is in a range (about 3W) which is easily accepted by the network camera, and the heat dissipation is better processed; secondly, the structure of the FPGA is fully parallel, so that the FPGA is very suitable for being used as an artificial intelligent algorithm; and thirdly, mature mass production chips are available in the FPGA chip market, repeated rapid reprogramming can be realized, and the development period is short so as to adapt to rapid iteration of the artificial intelligence algorithm. That is to say, the scheme of the network video monitoring device with the FPGA module has larger real touchdown property.

Drawings

The invention is described below with reference to the embodiments with reference to the drawings.

Fig. 1 shows a block diagram of a prior art network video monitoring apparatus.

Fig. 2 is a block diagram illustrating a configuration of a network video monitoring apparatus according to an embodiment of the present invention.

Fig. 3 is a block diagram illustrating a configuration of an FPGA module in a network video monitoring apparatus according to an embodiment of the present invention.

Detailed Description

The drawings are only for purposes of illustration and are not to be construed as limiting the invention. The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.

Fig. 1 shows a block diagram of a prior art network video monitoring apparatus. As shown in fig. 1, a conventional network video monitoring apparatus 100, such as a network camera, is mainly packaged with three main modules inside a housing 101, which are: a lens and sensor module 102, a video processing and transmission module 103, and a power and interface module 104. The lens and sensor module 102 is used to obtain raw video material, which is simply a camera. The video processing and transmission module 103 is used to process the raw video material and then transmit it over the network. The power and interface module 104 is used to supply power to the various modules and to provide a transmission interface.

According to the network video monitoring device provided by the invention, the FPGA module is added in the traditional network camera, so that the video data in the camera is processed by the FPGA module and then the data and information are determined to be transmitted back through the network.

As shown in fig. 2, a network video monitoring apparatus 200, such as a network camera, according to an embodiment of the present invention is similar to the conventional network video monitoring apparatus 100, and mainly includes three main modules, which are respectively packaged in a housing 201: a lens and sensor module 202, a video processing and transmission module 203, and a power and interface module 204. The lens and sensor module 202 is used to obtain raw video material, which is simply a camera. The video processing and transmission module 203 is used to process the raw video material and then transmit it over the network. The power and interface module 204 is used to supply power to the various modules and to provide a transmission interface.

Besides, the network video monitoring apparatus 200 according to the present invention further includes a Field Programmable Gate Array (FPGA) module 205, which is enclosed in the housing 201, and is configured to receive video data from the video processing and transmitting module 203, perform big data analysis processing on the received video data, and send the result of the big data analysis processing back to the video processing and transmitting module 203. The video processing and transmission module 203 determines data to be transmitted through the network according to the result of the big data analysis processing received from the FPGA module 205.

In a preferred embodiment, the big data analysis process performed by the FPGA module 205 is an artificial intelligence process. The result of the processing is structured information in the video data. More specifically, the artificial intelligence process is an image recognition and tracking process that results in the identification of tracked objects in the video data.

The invention is characterized in that an FPGA board card is added in the traditional network camera. The video input Interface comprises high-speed digital interfaces such as BT1120, RGMII (Reduced GMII (Gigabit Media Independent Interface), Reduced Gigabit Media Independent Interface) and the like. The communication Interface comprises a common digital Interface such as SPI (Serial Peripheral Interface), UART (Universal Asynchronous Receiver/Transmitter) and the like, and auxiliary reset control and power supply. The board card is structurally matched with the original structure of the camera. And an artificial intelligence algorithm is operated in the FPGA. Therefore, the upgrade from the traditional network camera to the artificial intelligent network camera is conveniently realized.

As can be seen from comparison between fig. 2 and fig. 1, the technical solution of the present invention has less change to the original architecture in the process of upgrading the network camera to the artificial intelligence network camera, and can realize industrialization quickly.

Fig. 3 is a block diagram illustrating a configuration of an FPGA module in a network video monitoring apparatus according to an embodiment of the present invention. In a preferred embodiment, the core chip of the FPGA module may adopt XC7Z020 of Xilinx, a DPU (Deep Learning Processing Unit) processor and an algorithm deployed on the processor are run inside the core chip, so that a high artificial intelligence algorithm performance can be realized under a low power consumption condition (for example, about 3W), a video of 25fps @1080p can be processed per second, more than 30 faces can be detected, modeled and recognized at the same time, and a face library of 10 ten thousand faces can be supported.

As shown in fig. 3, the FPGA module includes, in addition to the FPGA main chip, a power supply, a clock, a reset, memory particles, Flash for storing start and working programs, and a serial port UART for board debugging. In addition to this, there is a BT1120/RGMII/SPI/UART interface that interfaces with the audio video processing module of a conventional camera. Wherein a high speed digital interface, such as BT1120 or RGMII, is used to transfer video data from the video processing and transmission module 203 to the FPGA module. The FPGA module integrally serves as a coprocessor of the video processing and transmitting module 203, and the video processing and transmitting module 203 controls the work tasks and the work flows of the FPGA module and receives the work results of the FPGA module through a common digital interface, such as SPI or UART, and specifically, is used to transmit the results of big data analysis processing, such as artificial intelligence processing, from the FPGA module 205 to the video processing and transmitting module 203.

As previously mentioned, it will be appreciated by those of ordinary skill in the art that the results of the artificial intelligence process described above may be structured information in the video data. For example, in a preferred embodiment, the FPGA module 205 tracks a specific object in the video data through an artificial intelligence process, specifically an image recognition and tracking process, such as labeling the specific object with a box in the video for tracking. The result of the artificial intelligence process described above is a representation that can be reflected as structured information for transmission along with the video data.

Further, it will be understood by those of ordinary skill in the art that although the terms "video" or "video processing" are used herein, the actual data or processing also includes or processes audio data.

In a preferred embodiment, the size of the entire FPGA module 205 may be 5cm by 6cm, which can be easily built into a conventional webcam. It will be understood by those skilled in the art that heat dissipation of the video processing and transmission module (e.g., 103 of fig. 1) of a conventional network video surveillance device is achieved by adding a heat conductive gasket to the video processing chip through a raised boss at the bottom of the housing. In the present invention, the power consumption of the FPGA module 205 is slightly lower than that of the video processing and transmitting module, and the heat can be conducted to the top case by a similar method, so that the realizability is also strong. That is, the FPGA module 205 can dissipate heat by contacting the top-down boss in the housing plus the thermal pad. It will be understood by those skilled in the art that the terms "top," "bottom," "upward," "downward," and "upward" are used herein in a relative, rather than absolute, sense of spatial position.

The technical scheme of the invention is used as a core processing board card of the artificial intelligence network camera, video data is input by the BT1120 or a network interface, the artificial intelligence network camera is easy to be partially butted with the traditional network camera, the structure of the traditional network camera is changed to the minimum extent, and the artificial intelligence wings are installed on the traditional network camera at the minimum cost. The FPGA runs an artificial intelligent tracking and identifying algorithm of the security object, the frame number and various attribute information of the target object are output to a video processing part of the traditional network camera, and the traditional network camera determines the interception, transmission, overlapping display, storage and the like of the video according to the key information.

The FPGA module adopts an advanced artificial intelligence algorithm to analyze and process the video, so that the analysis accuracy is greatly improved, and the requirements of video analysis on environment and image quality are greatly reduced; the scheme can process a video of 25fps @1080p per second, can simultaneously perform detection, modeling and identification operations on more than 30 faces, can support 10 ten thousand face libraries, and can meet the security monitoring requirements in most scenes.

Through the scheme, the upgrading from the traditional network camera to the artificial intelligence network camera can be quickly realized, the artificial intelligence technology can better serve people, the labor cost is greatly saved, and the social operation efficiency is improved.

Because the FPGA module with low power consumption is used, compared with a graphic processor or Graphic Processing Unit (GPU) module which can be used as an alternative scheme, the power consumption of the FPGA module is within a range (about 3W) which is easily accepted by a network camera, and the heat dissipation is better processed.

On the other hand, the architecture of the FPGA is fully parallel, and is more suitable for application as an artificial intelligence algorithm than a possible ARM processor solution.

And compared with an AI special chip, the FPGA chip has the advantages that mature mass production chips are available in the market, repeated and rapid reprogramming can be realized, the development period is short, and the fast iteration of an artificial intelligence algorithm is very suitable.

Particularly, for the environment and problems faced by the preferred embodiment of the present invention, that is, for the network video monitoring apparatus, the video data is transmitted to the FPGA module with low power consumption and simple and convenient heat dissipation by using the high-speed digital interface, the FPGA module performs image recognition and tracking processing based on artificial intelligence on the massive video data, locks the target object, and returns the processing result to the video processing and transmitting module in the form of structured information, thereby solving the problems of time consumption, power consumption and financial consumption in the prior art and obtaining better effects than other alternative schemes.

Various embodiments and implementations of the present invention have been described above. However, the spirit and scope of the present invention is not limited thereto. Those skilled in the art will be able to devise many more applications in accordance with the teachings of the present invention which are within the scope of the present invention.

That is, the above examples of the present invention are only examples for clearly illustrating the present invention, and do not limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, replacement or improvement made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A network video monitoring apparatus, comprising:

the camera lens and sensor module is used for acquiring an original video material;

the video processing and transmitting module is used for processing the original video material and then transmitting the processed original video material through a network;

a power supply and interface module for supplying power to each module and providing a transmission interface,

it is characterized in that the preparation method is characterized in that,

the network video monitoring device further comprises:

a Field Programmable Gate Array (FPGA) module for receiving the video data from the video processing and transmitting module, performing image recognition and tracking processing on the received video data to obtain structural information in the video data, and sending the structural information in the video data back to the video processing and transmitting module, wherein the structural information includes the number of frames where the target object is located and a plurality of attribute information for identifying the tracked object in the video data,

the video processing and transmitting module intercepts video data according to the structural information received from the FPGA module, determines that only valuable video data related to the structural information are reserved, and displays the video data and the structural information in a superposition mode so as to transmit the video data through a network.

2. The network video monitor of claim 1, further comprising a high speed digital interface for transmitting video data from the video processing and transmission module to the FPGA module.

3. The network video monitoring device of claim 2, wherein the high speed digital interface is a BT1120 or RGMII interface.

4. The network video monitor of claim 2, further comprising a common digital interface for transmitting structured information in video data from said FPGA module to said video processing and transmission module.

5. The network video monitor of claim 4, wherein the common digital interface is an SPI or UART interface.

6. The network video monitoring device of claim 1, further comprising a housing for enclosing a lens and sensor module, the video processing and transmission module, the power and interface module, the FPGA module within the housing,

the video processing and transmission module is used for dissipating heat by contacting with a boss which is upwards raised at the bottom in the shell and a heat conducting gasket, and the FPGA module is used for dissipating heat by contacting with the boss which is downwards raised at the top in the shell and the heat conducting gasket.