CN111723725A

CN111723725A - Multi-dimensional analysis system based on video AI

Info

Publication number: CN111723725A
Application number: CN202010553712.1A
Authority: CN
Inventors: 魏巍
Original assignee: Xuzhou Wuyue Communication Technology Co ltd
Current assignee: Xuzhou Wuyue Communication Technology Co ltd
Priority date: 2020-06-17
Filing date: 2020-06-17
Publication date: 2020-09-29

Abstract

The invention belongs to the technical field of video AI analysis, and particularly relates to a multidimensional analysis system based on video AI, which comprises a hardware part and a software part, wherein the hardware part comprises a front-end monitoring device, a 5G transmission network, a rear-end intelligent video analysis system and a monitoring management center, the front-end monitoring device establishes data transmission connection with the rear-end intelligent video analysis system and the monitoring management center through the 5G transmission network, the data transmission connection is established between the rear-end intelligent video analysis system and the monitoring management center, and the key useful information in video sources is filtered out by means of the powerful data processing capacity of a computer, so that the camera not only becomes a human eye, but also the computer becomes a human brain. The intelligent video monitoring technology is one of the most advanced applications, and reflects the inevitable development trend of the future video monitoring system in all aspects of digitalization, intellectualization and diversification.

Description

Multi-dimensional analysis system based on video AI

Technical Field

The invention relates to the technical field of video AI analysis, in particular to a multidimensional analysis system based on video AI.

Background

The monitoring system is one of the most applied systems in the security system, the construction site monitoring system suitable for the market is a handheld video communication device, and video monitoring is the mainstream at present. From the earliest analog monitoring to the digital monitoring of the fire and heat in the previous years to the emerging network video monitoring, the change of the network coverage occurs.

With the progress of science and technology and the development of informatization, the traditional video monitoring system cannot meet the requirements of the industry.

Disclosure of Invention

The invention aims to provide a video AI-based multidimensional analysis system to solve the problem that the traditional video monitoring system cannot meet the requirements of the industry along with the progress of science and technology and the development of informatization in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme: a multi-dimensional analysis system based on video AI comprises a hardware part and a software part, wherein the hardware part comprises a front-end monitoring device, a 5G transmission network, a rear-end intelligent video analysis system and a monitoring management center, the front-end monitoring device establishes data transmission connection with the rear-end intelligent video analysis system and the monitoring management center through the 5G transmission network, and the rear-end intelligent video analysis system establishes data transmission connection with the monitoring management center;

the front-end monitoring equipment accesses the monitoring video stream to a video storage end through a 5G transmission network, the rear-end intelligent video analysis system calls the monitoring video stream in real time and detects and analyzes the monitoring video stream, and a detection result and alarm information are displayed to a user through a monitoring management center;

the software part comprises an intelligent monitoring retrieval platform, a characteristic database, an intelligent monitoring management platform and a characteristic identification server, wherein data transmission connection is established among the intelligent monitoring retrieval platform, the characteristic database, the intelligent monitoring management platform and the characteristic identification server;

the feature recognition server comprises an intelligent monitoring controller, the intelligent monitoring controller is connected with a video recognition controller, a video preprocessing manager and a video stream controller, the video recognition controller is connected with a video recognition module, the video preprocessing manager is connected with the video preprocessing module, the video stream controller is connected with a video source manager and a video recording manager, and the video source manager and the video recording manager are respectively connected with the video source processing module and the video recording module.

Preferably, front end supervisory equipment smart camera, intelligent wearing equipment and the robot of patrolling and examining.

Preferably, the video storage end is a local video storage end or a cloud video storage end.

Compared with the prior art, the invention has the beneficial effects that:

by means of powerful data processing capacity of the computer, useless or interference information in video picture is filtered out, and key useful information in the video source is automatically analyzed and extracted, so that the video camera becomes human eyes and the computer becomes human brain. The intelligent video monitoring technology is one of the most advanced applications, and reflects the inevitable development trend of the future video monitoring system in all aspects of digitalization, intellectualization and diversification.

Drawings

FIG. 1 is a diagram of the hardware architecture of the present invention;

FIG. 2 is a software logic block diagram of the present invention;

FIG. 3 is a schematic view of face identification of the present invention;

FIG. 4 is a diagram of the person location tracking of the present invention;

FIG. 5 is a flow chart of face recognition according to the present invention;

FIG. 6 is a face registration chart of the present invention;

FIG. 7 is a face recognition diagram of the present invention;

FIG. 8 is a tracking technique diagram of the present invention;

FIG. 9 is a field view of the oil line of the present invention;

FIG. 10 is an AI image recognition training diagram according to the present invention;

FIG. 11 is a diagram of a headgear wear detection application of the present invention;

FIG. 12 is a diagram of a smoking assay application of the present invention;

FIG. 13 is a diagram of an unbelted belt detection application of the present invention;

FIG. 14 is a diagram of a call-in behavior detection application of the present invention;

FIG. 15 is a process diagram of a video behavior detection-based analysis technique of the present invention;

FIG. 16 is a flow chart of a safety helmet detection algorithm of the present invention;

FIG. 17 is a flow chart of a call detection algorithm of the present invention;

FIG. 18 is a flow chart of a smoking detection algorithm of the present invention;

fig. 19 is a flow chart of seat belt behavior detection according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the description of the present invention, it is to be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention.

Referring to fig. 1-2, the present invention provides a technical solution: a multi-dimensional analysis system based on video AI comprises a hardware part and a software part, wherein the hardware part comprises a front-end monitoring device, a 5G transmission network, a rear-end intelligent video analysis system and a monitoring management center, the front-end monitoring device establishes data transmission connection with the rear-end intelligent video analysis system and the monitoring management center through the 5G transmission network, and the rear-end intelligent video analysis system establishes data transmission connection with the monitoring management center;

Further, front end supervisory equipment smart camera, intelligent wearing equipment and the robot of patrolling and examining.

Further, the video storage end is a local video storage end or a cloud video storage end.

The first embodiment is as follows: personnel identity recognition positioning system based on dynamic face recognition technology

The functional module jointly realizes the identification function of the personnel in the factory through the face recognition entrance guard arranged at the entrance and exit of each factory and the intelligent camera arranged in the operation area of the factory. The intelligent monitoring cameras in the large testing area of each laboratory 17 in the engineering center carry out identity recognition through a dynamic face recognition technology, and once foreign personnel are returned, alarm linkage is carried out. Fig. 3 is a schematic view of face identification.

The video monitoring is deployed in a full-coverage mode in each 17 large testing area of a laboratory in an engineering center, based on face recognition and pedestrian re-recognition technologies, a front-end camera captures pictures and transmits the pictures to a rear-end platform, and the rear-end platform performs comparison recognition and track tracking. The method has the advantages that the method can simultaneously detect the camera data of a plurality of different positions by uploading the pictures of specific characters in the deployment environment of the cameras, accurately find out the key positions of target character images captured by all the cameras to generate the tracking object, and generate the action space-time track of the tracking object by connecting the key positions, so that the purpose of cross-lens tracking is achieved. Fig. 4 is a person localization tracking diagram.

The face recognition system utilizes a distributed clustering technology, a neural network-based deep learning technology and a mass data storage big data computing technology to realize real-time video monitoring of images and videos. The face recognition is a process of comparing the information of the face to be detected with the information in the database to complete matching. The face information to be detected can be pictures, pictures captured by a camera in real time, and also can be stored video information. A complete face recognition process comprises four processes of face detection, image preprocessing, feature extraction and face matching. Fig. 5 is a flow chart of face recognition.

Face collection

Video face collection

The system can realize video monitoring of a specific monitoring area through the front-end acquisition equipment, and realize a video acquisition function. The system supports multi-image source collection, and comprises multiple collection modes such as an IP camera, a USB camera, a CCTV camera and a video file. The collected video can be used for real-time face comparison or stored through video compression. All cameras in the area can be associated by combining information such as time, space, target characteristics and the like, and cross-camera tracking can be performed on a specific target.

Video face snapshot and storage

The system has the functions of capturing and storing face images in videos, and captured face images are automatically cut to reduce the pressure of network resources occupied by video transmission, automatically numbered and stored through a face detection technology. The captured faces may be efficiently indexed for querying and access.

Face registration

And calling the real-time video, positioning the face when a person enters a monitoring area, selecting a face image after snapshot, and registering. And the personnel registry can be customized in the background to carry out batch registration. Fig. 6 is a face registration diagram.

Face comparison and recognition

Fig. 7 is a face recognition diagram.

Through comparison with a historical library, whether the personnel appears in the control camera or not, the appearance time and other information can be determined, and the personnel and the suspicious positions can be quickly positioned. The dynamic video face comparison and analysis node is connected with each monitoring camera distributed and controlled at the front end, so that the functions of identity verification comparison, video recording and real-time alarming of personnel can be effectively realized, and an effective and real-time alarming mechanism is provided.

The video face detection and identification means that whether a face exists or not is judged in a dynamic scene and a complex background, and the face is separated. Under the video monitoring environment, due to the changing factors such as motion, posture, shielding and illumination, the definition of the human face is often insufficient, the system can detect the human face of a real-time video and a video recording, can remove a background area, accurately position the human face image in the video, and can adapt to the conditions of different human face sizes, complex background changes and the like.

The human face track tracking function based on the electronic map can associate all cameras in the area by combining information such as time, space, target characteristics and the like, and perform cross-camera tracking on a specific target; the movement track of the target on the map can be visually displayed, and the surrounding monitoring video resources can be visually displayed only by clicking the corresponding position on the map. And a scientific video decision basis is provided for visual command and scheduling, and resource sharing and cooperative combat are realized. FIG. 8 illustrates a tracking technique.

The system supports automatic tracking, calibration, clustering and optimization of the faces appearing in the video, and can be used for people counting at the later stage; all cameras in the area are associated by combining information such as time, space, target characteristics and the like, and cross-camera tracking is carried out on a specific target identified by the human face; the movement track of the target on the map can be visually displayed, and the surrounding monitoring video resources can be visually displayed only by clicking the corresponding position on the map.

The personnel identity recognition and positioning system based on the dynamic human face integrates a deep learning technology and big data training on the basis of the traditional computer vision algorithm, realizes the functions of static and dynamic object recognition and two-dimensional positioning with high precision and high real-time performance, uses a camera to collect positioning information, and does not need a positioned object to carry positioning labels and other equipment.

Example two: oil leakage real-time analysis alarm system based on artificial intelligence technology

Functional design

For daily maintenance and overhaul of a factory, oil leakage and air leakage of a pipeline are very important links for operation inspection. The oil leakage is difficult to find in time during the operation of the equipment, because the leakage process is not detectable and discoverable in time, the daily routing inspection has time intervals and individual errors, and the leakage condition is difficult to be found in time. It is very necessary to adopt intelligent video automatic oil leakage monitoring.

The method is characterized in that the lubricating oil leakage condition is detected in real time based on a deep learning image recognition technology, a camera is adopted to shoot common oil leakage positions in real time, real-time analysis is carried out through background intelligent analysis service, and once abnormal conditions occur, warning is carried out to remind a manager to carry out processing rapidly.

The method is characterized in that spray forms are often used as main features for high-pressure pipeline oil injection, spray in a monitoring area can be detected and early warned in an intelligent video analysis mode, the fact that spray generation points are generally fixed and the generation area is limited is considered, therefore, an AI image recognition algorithm is adopted to monitor and analyze the pipeline area, and once spray form scenes are detected, an alarm is triggered to remind operating personnel to process the spray forms. Fig. 9 is an oil duct field view.

The method comprises the steps of aiming at the condition of oil injection of a pipeline, obtaining a special material difficultly, building a pipeline oil circuit field environment through simulation by using an AI image recognition algorithm pipeline oil injection condition training material, obtaining the training material through artificial scene reproduction, marking material data with a certain scale and high quality through data, obtaining a high-precision network model through a training neural network, and deploying the high-precision network model in a production environment for intelligent detection and recognition of oil leakage.

Description of the technology

Image recognition is an important field of artificial intelligence, which refers to the processing, analysis and understanding of images by computers. To identify various different patterns of objects and objects. The deep learning-based AI image recognition algorithm training process is divided into three stages, namely data preparation, model development and model application, as shown in the following figure. Fig. 10 is an AI image recognition training diagram.

Preparing data: and (4) building an oil pipeline scene, simulating an oil leakage condition, and collecting oil leakage image materials.

Model development: the method comprises the steps of selecting a training sample, excavating and selecting features, training a model by using an algorithm, evaluating the performance of the model by using test data, and finally obtaining the model meeting the requirements through continuous optimization.

Application of the model: according to different production system environments, the model can be deployed on a cloud end, an all-in-one machine or terminal equipment. Due to the fact that the operating environment and hardware are different, specific performance optimization needs to be carried out on the model, and therefore delay of the system is reduced, the requirement for resources is reduced, and the operating cost of the system is reduced.

Example three: personnel behavior analysis and alarm system based on artificial intelligence technology

Functional design

In order to implement relevant legal regulations of safety production and realize deep fusion of artificial intelligence and safety production operation, the personnel behavior analysis and alarm system based on the artificial intelligence technology aims to provide technical means such as visual intelligent management, dangerous accident prediction and the like, so that the management efficiency of a construction site is improved, the cost of a building unit is reduced, accidents are reduced, and the true win-win situation of society, enterprises and staff is realized.

The personnel behavior analysis and alarm system based on the artificial intelligence technology applies the intelligent video analysis and deep learning neural network technology, performs real-time analysis, identification, tracking and alarm on whether a person in a creep production area wears a safety helmet, smokes, makes a call, does not fasten a safety belt and the like through video monitoring, directly performs real-time early warning on dangerous behaviors of not wearing the safety helmet, smokes, makes a call and does not fasten the safety belt through video real-time analysis and early warning without depending on other sensors, chips and labels, stores alarm screenshots and videos into a database to form a report, and meanwhile pushes the alarm information to related managers. Fig. 11 is a diagram of a helmet wearing detection application. Figure 12 is a diagram of smoking detection applications. Fig. 13 is a diagram of an unbelted belt detection application. Fig. 14 is a diagram of a call-in behavior detection application.

The detection algorithm of the AI video behavior detection analysis technology is generally divided into two parts, namely training and identification. In the training part, carrying out random variation (including preprocessing such as resolution conversion and text superposition) on videos in a reference video library, then extracting local depth features from video key frames, and fusing the local depth features by utilizing a multi-clue semantic fusion strategy to form a frame feature library;

in the identification part, the key frames of the video to be detected are subjected to the same operation to extract features, then the features are compared with the feature library through a feature index technology and a reference video is positioned, and a corresponding behavior video clip is positioned by using a video time alignment technology. Fig. 15 is a process diagram of a video behavior detection-based analysis technique.

Firstly, detecting the face position of an operator in a field video by using a deep learning algorithm, and estimating a potential area of the safety helmet according to the relation between the safety helmet and the face; then, enhancing the potential area image of the safety helmet, and extracting a characteristic vector of the sample; whether a safety helmet is arranged above the face is judged by the classifier, and then real-time detection and early warning of wearing behaviors of the safety helmet of construction personnel are achieved. Fig. 16 is a flow chart of a helmet detection algorithm.

The call behavior detection based on the AI video analysis technology utilizes the positioning of a face area and two hand areas to judge whether the behavior of holding the phone exists by detecting whether the two hands are lifted and attached to the two sides of the face. Fig. 17 is a flow chart of a call detection algorithm.

Smoking behavior detection based on AI video analysis techniques may be achieved by utilizing motion characteristics of the hand and its specific hand gesture characteristics. Therefore, the smoking behavior detection and recognition are realized by detecting the moving target human hand from the image sequence containing the human, extracting the hand area image, tracking, further analyzing the gesture characteristics and recognizing. Figure 18 is a flow chart of a smoking detection algorithm.

The method for detecting the behavior of the unbelted safety belt based on the AI video analysis technology utilizes the approach characteristic of gradually reducing the ROI area of a video image, firstly, a license plate and a vehicle window are positioned, then, the position of a driver is detected through face detection, and finally, the safety belt is detected near the position of the driver. Fig. 19 is a flow chart of seat belt behavior detection.

The application of the artificial intelligence video analysis technology realizes the timely detection, early warning and continuous tracking of various safety events in a video monitoring area. Can effectively prevent and stop various potential safety hazards.

While there have been shown and described the fundamental principles and essential features of the invention and advantages thereof, it will be apparent to those skilled in the art that the invention is not limited to the details of the foregoing exemplary embodiments, but is capable of other specific forms without departing from the spirit or essential characteristics thereof; the present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein, and any reference signs in the claims are not intended to be construed as limiting the claim concerned.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims

1. A multi-dimensional analysis system based on video AI, comprising a hardware part and a software part, characterized in that: the hardware part comprises front-end monitoring equipment, a 5G transmission network, a rear-end intelligent video analysis system and a monitoring management center, wherein the front-end monitoring equipment establishes data transmission connection with the rear-end intelligent video analysis system and the monitoring management center through the 5G transmission network, and the rear-end intelligent video analysis system establishes data transmission connection with the monitoring management center;

2. The video-based AI multidimensional analysis system of claim 1, wherein: the front end monitoring equipment intelligent camera, intelligent wearing equipment and the inspection robot.

3. The video-based AI multidimensional analysis system of claim 1, wherein: the video storage end is a local video storage end or a cloud video storage end.