WO2016004673A1

WO2016004673A1 - Intelligent target recognition device, system and method based on cloud service

Info

Publication number: WO2016004673A1
Application number: PCT/CN2014/085640
Authority: WO
Inventors: 王向恒
Original assignee: 王向恒
Priority date: 2014-07-11
Filing date: 2014-08-30
Publication date: 2016-01-14
Also published as: CN104159071A; CN107005679A; CN107005679B

Abstract

Disclosed are an intelligent target recognition device, system and method based on cloud service. The intelligent target recognition device classifies video data according to a data classification method, and sends a recognition result to a monitor terminal; video characteristic information incorrectly recognized is fed back to a cloud server by the monitor terminal according to user selection; the cloud server processes the video characteristic information, trains to generate a new data classification method, and sends an update notification to the intelligent target recognition device; the intelligent target recognition device receives the notification, downloads and updates a data classification method configuration file. The present invention provides a user with an intelligent target recognition device applicable to the monitored environment of the user by continuously obtaining user feedback and dynamically updating the data classification method, thus effectively reducing video monitoring cost and false alarm rate, and improving recognition accuracy of an intelligent video monitoring system.

Description

Intelligent target recognition device, system and method based on cloud service

Technical field

The invention relates to a smart target recognition device based on cloud service, and relates to an intelligent target recognition system and a corresponding intelligent target recognition method using the device, and belongs to the technical field of video surveillance.

Background technique

The existing video surveillance system has simple motion detection and alarm functions, but it does not have the functions of target recognition, behavior understanding, event recognition, etc., but the real security protection requires people to observe and analyze images in real time. Get the security evaluation of the scene. In order to improve the intelligence of the video surveillance system, people are freed from boring on-site surveillance. Many scholars combine optoelectronic digital image processing and analysis, pattern recognition, computer vision, artificial intelligence and other technologies with video surveillance to propose intelligent visual surveillance. The research direction has carried out in-depth research in the fields of detection, tracking, classification, recognition, target behavior understanding and event recognition of moving targets.

The monitoring effect of the video surveillance system is directly related to the data classification method adopted, and the data classification method is obtained through data sample training analysis. In the prior art, there are usually two ways to obtain data samples: one is through a public database. Referring to FIG. 1 , in this mode, since the data samples are greatly different from the actual usage scenarios, it is easy to cause false negatives and false positives in the actual application process; the second is to directly pass the user's monitoring video. In this way, the real-time monitoring video is directly transmitted to the cloud server, and the cloud server performs training analysis on the video feature data. In this mode, although the video data comes from the real application scenario, it has the following drawbacks: First, the cloud server synthesizes all data in different occasions, that is, all users share a data classification method. The video surveillance system is usually used in a specific scenario. The shared data classification method will make the detection accuracy random. Secondly, the cloud server needs to process and store a large amount of invalid video information (such as no abnormality in the scene). Will consume a lot of bandwidth and server costs.

For example, US Patent Application Publication No. US2014043480 discloses a video surveillance system and method, in which a front-end data acquisition device collects video images and transmits video image data to a front-end access device; the front-end access device transmits the front-end data acquisition device. The video image data is transmitted to the cloud system; the cloud system analyzes the video image data, and generates an alarm when the behavior of the target in the video image collected by the front-end data acquisition device is abnormal. The invention analyzes and processes video image data through a cloud system, and all users share a video analysis server, and the accuracy of the monitoring may have a certain error, and the cost of the server is relatively high.

Summary of the invention

In view of the deficiencies of the prior art, the primary technical problem to be solved by the present invention is to provide a cloud object-based intelligent object recognition device.

Another technical problem to be solved by the present invention is to provide an intelligent target recognition system using the above-described intelligent target recognition device.

Another technical problem to be solved by the present invention is to provide an intelligent target recognition method based on the above-described intelligent target recognition system.

In order to achieve the above object, the present invention adopts the following technical solutions:

A smart object recognition device based on a cloud service, comprising a video analysis module, a data classification module, a storage module, a communication module and a control module; wherein the control module is responsible for information interaction between the modules;

The video analysis module performs analysis processing on the video data transmitted by the communication module; the data classification module determines whether the monitoring environment is abnormal according to the image information processed by the analysis module, and sends alarm information to the monitoring terminal through the communication module; the storing The module is used to store the collected video data;

The control module receives a video viewing instruction of the monitoring terminal on the one hand, notifies the storage module to transmit the video data to the monitoring terminal, and downloads and updates the configuration file in the data classification module through the communication module.

An intelligent target recognition system based on a cloud service, comprising the above intelligent target recognition device, a video collection device, a monitoring terminal, and a cloud server; the smart target recognition device and the video collection device, the monitoring terminal, and the cloud server respectively Connected through the network;

The video collection device collects video information of the monitoring environment in real time;

The smart target recognition device is used for video analysis and data classification, and sends the recognition result and the video to the monitoring terminal;

The monitoring terminal sends the identification result that is determined to be incorrect and the data sample of the video refinement to the cloud server;

The cloud server sends the data sample of the recognition result and the video refinement to the terminal device as a data sample, and trains to generate a configuration file for updating the data classification of the smart object recognition device.

An intelligent target recognition method based on cloud service is implemented based on the above intelligent target recognition system, and includes the following steps:

The intelligent target recognition device classifies the video data according to the data classification method, and transmits the recognition result to the monitoring terminal;

The monitoring terminal feeds back the misidentified video feature information to the cloud server according to the user's selection;

The cloud server processes the video feature information, trains to generate a new data classification method, and sends an update notification to the smart target recognition device;

The smart target recognition device receives the notification, downloads and updates a data classification method configuration file.

The present invention separates the actual use of the data classification method from the data training process. In use, the smart target recognition device is arranged in the monitoring environment, and the user sends the misclassified data classification result to the Cloud Server. The cloud server only needs to train and generate a data classification method for updating the smart target recognition device according to the video feature data fed back by the user. Therefore, the present invention is particularly suitable for a separate user monitoring environment, which can effectively reduce the false alarm rate and improve the recognition accuracy of the video monitoring system.

DRAWINGS

Figure 1 is a flow chart showing the generation and use of a data classification method;

2 is a schematic structural diagram of a smart target recognition device provided by the present invention;

3 is a schematic structural diagram of an intelligent target recognition system provided by the present invention;

FIG. 4 is a flowchart of a smart target recognition method provided by the present invention.

detailed description

The technical content of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

Referring to FIG. 2, the smart target recognition device provided by the present invention is located in a user's monitoring environment and is used in conjunction with the video capture device. The smart target recognition device analyzes the video collected by the video capture device, and determines whether the current environment has an abnormal situation. When there is an abnormal situation, an alarm notification is pushed to the user through the network. The device can also automatically update the data classification method when there is a configuration update. In an embodiment of the invention, the smart object recognition device comprises a video analysis module, a data classification module, a storage module, a communication module, and a control module. The control modules are each connected to each module. The communication module is respectively connected to the video analysis module and the data classification module.

The video analysis module is used for analyzing and processing the collected video data, and specifically includes three units of moving target detection and extraction, target segmentation and feature extraction. These units can be implemented in software or firmware. Among them, the moving target detection and extraction unit can adopt algorithms such as frame difference method, optical flow method and dynamic adaptive background method, so that the background and moving objects can be distinguished. After the moving object detection and extraction unit extracts the detected object, the target segmentation unit divides the target. The target segmentation can be performed by the Otsu method (maximum inter-class variance method), iterative method, maximum entropy method, and the like. The feature extraction unit extracts feature information of the image, such as a color, a shape, a motion trajectory, and the like, by tracking the segmented target. The image feature information is usually saved in the form of a feature vector.

The data classification module determines whether the monitoring environment is abnormal according to the image information processed by the analysis module. The data classification module requires a built-in initialized data classification method configuration file (usually in resident memory or firmware). The initialized data classification method configuration file is obtained from a large number of benchmark sample data. The data classification method can be understood as a mapping relationship, which automatically maps the input feature vector to +1 or -1. In the present invention, +1 and -1 respectively represent two types of recognition results of "important events intrusion" and "non-significant events in motion". After the data classifier identifies which category the current video belongs to, it sends the recognition result to the user's monitoring terminal. Through continuous feedback from users, cloud server training generates new data classification methods. This method is used for the configuration file in the updated data classification module. Such a data classification module can continuously update the data classification method. And the new data classification method is Adapt to the current monitoring environment, so the new classification method for identification and classification can effectively reduce the false positive rate.

The storage module is used to store the collected video data, which is convenient for the user to view and play back. The storage module can be implemented by using Flash Memory, DDR SDRAM, or the like.

The communication module is used for information interaction between the smart target recognition device and the video capture device, the monitoring terminal, and the cloud server. The communication module includes a video access unit and a data interaction unit. The video access unit is configured to access the collected video and send the video to the video analysis module and the storage module. The video access unit is a video decoding circuit, and is used for decoding video data collected by the video capture device. The data interaction unit is used for data interaction between the smart target recognition device and the monitoring terminal, and sends the classification result of the data classification module to the monitoring terminal, and simultaneously receives the video viewing information sent by the monitoring terminal; The information is exchanged between the target recognition device and the cloud server. The cloud server notifies the smart target recognition device to update the classification profile in the data classification module by training the video feature data. The data interaction unit may adopt any one of wired network communication or wireless network communication. For example, the wired network may adopt an Ethernet interface, and the wireless network communication may be WIFI, 3G/4G, or the like.

The control module is a core functional module of the intelligent target recognition device, and controls data communication and interaction between the respective modules. The control module can be implemented by a single chip microcomputer or a microcontroller (MCU), etc., after receiving the video viewing instruction of the monitoring terminal, notifying the storage module to transmit the video data to the monitoring terminal, and downloading the cloud server through the communication module. Update the configuration file of the data classification method in the data classification module.

For a video analysis module, a data classification module, a control module, etc., those skilled in the art can understand that the above embodiments can be implemented in software, hardware or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated design hardware. One of ordinary skill in the art will appreciate that the methods and systems described above can be implemented using computer-executable instructions and/or embodied in processor control code, such as a carrier medium such as a magnetic disk, CD or DVD-ROM, such as a read-only memory. Such code is provided on a programmable memory (firmware) or on a data carrier such as an optical or electronic signal carrier. The system of this embodiment and its components can be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, and the like. It can also be implemented by software executed by various types of processors, or by a combination of the above hardware circuits and software such as firmware.

Referring to FIG. 3, the present invention also provides a smart object recognition system based on cloud service using the above intelligent target recognition device. The smart target recognition system further includes a video capture device, a monitoring terminal, and a cloud server. The video capture device is a camera. The camera is wired or The wireless Internet access method is connected with the intelligent target recognition device, and is used for real-time collecting video information of the monitoring environment, and transmitting the collected video to the intelligent target recognition device for processing.

The monitoring terminal is respectively connected to the smart target recognition device and the cloud server through the network, and is used for viewing the monitoring situation by the user. The monitoring terminal receives the recognition result information sent by the smart target recognition device, and sends the determined video feature information to the cloud server for training. The monitoring terminal in the present invention is a smart terminal device with human-computer interaction function, including a smart phone, a computer, and a TV with interactive functions.

The cloud server is respectively connected to the intelligent target recognition device and the monitoring terminal through the network, and is used for training the video feature data, and dynamically updating the data classification method according to the information fed back by the user. The cloud server includes a video analysis server, a storage server, and a control server. The storage server includes a video storage area, a feature database, and a data classification method configuration file. The feature database includes a human body data feature library and a non-human body feature database. The analysis server extracts the feature information according to the video segment sent by the monitoring terminal, and stores the feature information and the classification result into the feature database as a new sample. The control server is configured to parse the command sent by the monitoring terminal, and train the data according to the updated feature database, and generate a new data classification method configuration file, and send an update notification to the smart target recognition device.

Referring to FIG. 4, the smart target recognition method based on the above intelligent target recognition system provided by the present invention specifically includes the following steps:

Step 1: The intelligent target recognition device analyzes and identifies the acquired surveillance video processing, and sends an alarm message to the monitoring terminal held by the user through the network. The intelligent target recognition device is arranged in the monitoring environment to detect and determine whether there is an abnormality in the current environment. Once the monitoring environment is abnormal, the analysis module in the device performs target detection, target segmentation, and feature extraction on the currently acquired video information. The data classification module identifies and classifies the acquired feature information according to the configuration data classification method, and sends the identification result to the monitoring terminal. According to the data classification method, the image feature vectors are classified and classified to determine whether the current monitoring video is an important event or a non-critical event. The intelligent target recognition device calculates the recognition result and pushes the alarm information to the monitoring terminal of the user through the network.

In step 2, the monitoring terminal receives the alarm notification of the smart target recognition device. The user views the video according to the recognition result, and determines whether the recognition result is correct; if the recognition result is incorrectly determined, the user will modify the recognition result of the video segment, and transmit the feature data extracted by the video segment and the modified recognition result to the On the cloud server.

The two types of recognition results are sent to the user's monitoring terminal with different warning signs. After receiving the warning sign, the user sends a video viewing command to the smart target recognition device according to the warning flag. The smart target recognition device converts the video transmitted by the storage module according to the device information transmitted by the monitoring terminal, and sends a monitoring video suitable for the terminal to the monitoring terminal. If the user clicks on the video of the two events and finds that the video clip does not match the recognition result, it is not important that the event marked as important is not important, or that the mark of a certain class is important, the user can modify the mark of the current event. . When a mobile terminal processes an event, it usually marks the important event as 1 and the non-important as -1. When the user modifies the flag, it changes 1 to -1 or -1 to 1. After the modification is completed, the monitoring terminal automatically uploads the feature data extracted by the video segment and the corresponding recognition result (1 or -1) to the cloud server.

In the present invention, the user's feedback data is used as a training sample for generating a new data classification method. Compared with the prior art that uses the data in the public database for training analysis, the data samples in the present invention are consistent with the actual usage scenarios, so that the data classification method is more suitable for the actual application scenario. In addition, the feedback data of each user is only used as a data sample for generating a data classification method of the current monitoring environment. Therefore, the generated data classification method is only applicable to the current monitored real scene, and the accuracy of video monitoring can be effectively improved.

Step 3: The cloud server receives and processes the video feature information sent by the monitoring terminal, and the video feature information is trained as sample data to generate a new data classification method, and the smart target recognition device is notified to update the configuration file.

First, the cloud server stores the video feature information in a separate storage area of each user according to the terminal information. Before the smart target recognition system works, the user sends the registration information to the cloud server through the monitoring terminal. The cloud server separately establishes a storage area for storing the video feature database and the data classification method according to the user name registered by each user. When the monitoring terminal sends the video feature information, the monitoring information of the user is simultaneously sent to the cloud server. The cloud server stores the feature information of the video refinement into its corresponding storage area according to the user name registered by the user. When generating the data classification method, only the training samples of the user storage area are used, and the stored information of other users is not involved. Therefore, the cloud server of the present invention generates a data classification method for the specific scene of the user, rather than a data classification method for all users.

In the prior art, a cloud server directly connected to a webcam needs to process all monitoring information of the user, including abnormal and normal video data. The cloud server in the present invention only stores video information that the user has marked important, and does not store the video in the normal state where there is no object moving in the scene, and the video that is dynamic but not important, so that the cloud occupied by a large amount of invalid information can be effectively reduced. The server has a lot of bandwidth and cost.

Secondly, the training samples are stored in the corresponding feature database according to the classification marks. At the time of classification, the feature vector and its classification mark are used as inputs and are operated in a simple classification function. The classification function divides the feature vector into positive samples or negative samples based only on the recognition results (+1 and -1). The feature database includes a human body database and a non-human body database. At the time of system initialization, the feature database contains the feature data that generates the initial classification method. The new feature data fed back by the user is merged with the initial feature data. If the user marks the importance, these features are added to the body feature database as positive samples, otherwise they are added as negative samples to the non-human body database. Then, according to the new feature database after the merger, a new data classification method is generated.

Then, based on the updated feature database, the data is trained and a new data classification method is generated. More The new sample information is calculated by the training data classification method, which can optimize the edge of the decision boundary and reduce the classification error. The training data classification method of the invention adopts SVM (Support Vector Machine), so for the outlier data samples, a penalty factor is also needed to be corrected, and finally a new data classification method for accurately distinguishing the two types of samples is generated. The latest profile of the data classification method is sent over the network to the smart target recognition device.

In step 4, the smart target recognition device downloads the data classification configuration file and updates the data classification method. Each smart target recognition device has a separate ID. When the user sends the registration information to the cloud server through the monitoring terminal, the cloud server binds the smart target identification device ID to the registered user name according to the registration information. After the cloud server generates a new data classification method, the smart target identification device ID bound to the user name is found by the user name registered by the user. The cloud server transmits the update notification to the designated smart target recognition device by the ID of the device. And send an update request. The smart target recognition device automatically reads the configuration file in the cloud server and updates the data classification method in the data classification module according to the update information of the cloud server. The smart target recognition device continues to process the surveillance video data according to the updated data classification method.

In summary, the present invention separates the actual use of the data classification method from the training process of the data, and the user provides the video features required for the training data. This continuously updated data classification method is adapted to each user's monitoring environment. The specific intelligent target recognition device and the surveillance camera are configured in the monitoring environment, and the monitoring terminal obtains the data classification result from the network, and sends the result of the analysis error to the cloud server through the network. The cloud server adds the video feature data fed back by the user to the training sample, and trains to generate a new data classification method according to the updated data sample. The new data classification method is continuously optimized for a user's monitoring environment, which can effectively reduce the false alarm rate and improve the recognition accuracy of the intelligent video surveillance system.

The cloud service-based intelligent object recognition device, system and method provided by the present invention are described in detail above. Any obvious changes made to the present invention without departing from the spirit of the invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal liabilities.

Claims

A smart object recognition device based on a cloud service, comprising: a video analysis module, a data classification module, a storage module, a communication module, and a control module; wherein the control module controls information interaction between the modules;

The video analysis module performs analysis processing on the video data transmitted by the communication module; the data classification module determines whether the monitoring environment is abnormal according to the image information processed by the analysis module, and sends alarm information to the monitoring terminal through the communication module; the storing The module is used to store the collected video data;

The control module receives a video viewing instruction of the monitoring terminal on the one hand, notifies the storage module to transmit the video data to the monitoring terminal, and downloads and updates the configuration file in the data classification module through the communication module.
The intelligent object recognition device according to claim 1, wherein:

The video analysis module includes a moving target detection and extraction unit, a target segmentation unit, and a target feature extraction unit that are sequentially connected; wherein, after the moving target detection and extraction unit extracts the detected object, the target segmentation unit targets the target Segmentation is performed, and the feature extraction unit tracks the segmented target.
The intelligent object recognition device according to claim 1 or 2, wherein:

The data classification module internally initializes the data classification method configuration file in firmware.
A smart object recognition system based on a cloud service, comprising: a video capture device, a monitoring terminal, a cloud server, and the smart object recognition device according to any one of claims 1 to 3; The video collection device, the monitoring terminal, and the cloud server are connected through a network;

The video collection device collects video information of the monitoring environment in real time;

The smart target recognition device is used for video analysis and data classification, and sends the recognition result and the video to the monitoring terminal;

The monitoring terminal sends the identification result and the video that are determined to be incorrect to the cloud server;

The cloud server sends the recognition result and the video as a data sample, and the training generates a configuration file for updating the data classification of the smart object recognition device.
The intelligent object recognition system according to claim 4, wherein:

The monitoring terminal is a smart terminal device with human-computer interaction function, including a smart phone, a computer or a television.
The intelligent object recognition system according to claim 4 or 5, wherein:

The cloud server includes a video analysis server, a storage server, and a control server; wherein the storage server includes a video storage area, a feature database, and a data classification method configuration file;

The analysis server is configured to send a video analysis to the monitoring terminal, and use the analysis result as a new number. According to the sample stored in the feature database;

The control server trains and generates a new data classification method according to the feature database, and notifies the smart target recognition device to update the data classification method configuration file.
A smart object recognition method based on a cloud service, which is implemented by the intelligent object recognition system according to any one of claims 4 to 6, which comprises the following steps:

The intelligent target recognition device classifies the video data according to the data classification method, and transmits the recognition result to the monitoring terminal;

The monitoring terminal feeds back the misidentified video feature information to the cloud server according to the user's selection;

The cloud server processes the video feature information, trains to generate a new data classification method, and sends an update notification to the smart target recognition device;

The smart target recognition device receives the notification, downloads and updates a data classification method configuration file.
The intelligent object recognition method according to claim 7, wherein:

Before the smart target recognition system works, the user sends registration information to the cloud server through the monitoring terminal, and the cloud server binds the smart target recognition device to the monitoring terminal according to the registration information.
The intelligent object recognition method according to claim 8, wherein:

The cloud server searches for the bound smart target recognition device according to the registration information, and sends the update notification to the smart target recognition device.
The smart object recognition method according to any one of claims 7 to 9, wherein the processing, by the cloud server, the video feature information further comprises the following steps:

Depositing the video feature information into a storage area of the user;

Dividing the feature information into a positive sample or a negative sample according to the classification mark, and storing the new sample in the feature database;

Based on the updated feature database, the training generates a new data classification method.