CN113723184A - Scene recognition system, method and device based on intelligent gateway and intelligent gateway - Google Patents
Scene recognition system, method and device based on intelligent gateway and intelligent gateway Download PDFInfo
- Publication number
- CN113723184A CN113723184A CN202110839664.7A CN202110839664A CN113723184A CN 113723184 A CN113723184 A CN 113723184A CN 202110839664 A CN202110839664 A CN 202110839664A CN 113723184 A CN113723184 A CN 113723184A
- Authority
- CN
- China
- Prior art keywords
- scene
- video stream
- stream data
- intelligent gateway
- scene recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Abstract
The application relates to a scene recognition system, a scene recognition method, a scene recognition device, an intelligent gateway and a storage medium based on the intelligent gateway, and specifically comprises the intelligent gateway and a camera device; the camera shooting device is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified; the intelligent gateway is used for acquiring a scene image to be identified from the camera equipment; and inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier, and sending the video stream data to a corresponding anomaly detection server according to the scene category identifier. The intelligent gateway identifies the scene image to be identified in the video stream data acquired by the camera equipment to obtain a scene identification result; and the video stream data is sent to the corresponding anomaly detection server according to the scene category identification in the identification result, so that the identification, classification and sending processing of the video stream data are realized, and the scene identification efficiency is improved.
Description
Technical Field
The application relates to the technical field of power grid management, in particular to a scene recognition system, a scene recognition method, a scene recognition device, an intelligent gateway and a storage medium based on the intelligent gateway.
Background
With the development of power grid management technology, the construction of monitoring systems in various business fields of power enterprises is developed; the continuous expansion of an electric power system, the distribution of power transmission and distribution lines and corresponding equipment is wider and wider, a plurality of power equipment do not have a data transmission function, and data of the power equipment need to be acquired through a plurality of camera equipment.
However, the monitored power equipment has many types and quantities, and the data transmission quantity of the camera equipment is huge, so that data of different scenes need to be processed respectively by a plurality of servers at the same time; therefore, a scene recognition system based on an intelligent gateway is also needed to improve the efficiency of scene recognition.
Disclosure of Invention
In view of the above, it is necessary to provide a scene recognition system, a scene recognition method, a scene recognition device, an intelligent gateway and a storage medium based on an intelligent gateway.
A scene recognition system based on an intelligent gateway comprises: the system comprises an intelligent gateway and camera equipment; the intelligent gateway is in communication connection with the camera equipment;
the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified;
the intelligent gateway is used for acquiring the scene image to be identified from the camera equipment; inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier; and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
In one embodiment, the intelligent gateway is further configured to control the camera device to rotate in response to an instruction of the terminal device, so that the camera device collects video stream data of different scenes to be recognized.
In one embodiment, the system further comprises: a database;
the database is in communication connection with the intelligent gateway and is used for receiving video stream data sent to the anomaly detection server by the intelligent gateway and storing the video stream data.
In one embodiment, the anomaly detection server is further configured to obtain video stream data in the database; carrying out anomaly detection on a scene image in the video stream data; and if the scene image is detected to be abnormal, generating alarm information and sending the alarm information to a preset terminal.
In one embodiment, the pre-constructed scene recognition model is obtained by training a scene recognition model to be trained, which is constructed based on a YOLO algorithm, for multiple times; the scene recognition model to be trained comprises a convolution layer, a pooling layer and a full-connection layer.
A scene recognition method based on an intelligent gateway comprises the following steps:
acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
In one embodiment, the pre-constructed scene recognition model is obtained by:
acquiring a sample image set; the sample image set comprises sample images of a plurality of scene categories;
inputting the sample image set into a scene recognition model to be trained, performing iterative training on the scene recognition model to be trained by adopting a loss function, and obtaining a loss value according to the class probability corresponding to each scene class;
and adjusting the model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as the pre-constructed scene recognition model.
An intelligent gateway based scene recognition apparatus, the apparatus comprising:
the image acquisition module is used for acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
the scene recognition module is used for inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result corresponding to the scene image to be recognized;
and the data sending module is used for sending the video stream data to the corresponding terminal equipment according to the scene identification result.
An intelligent gateway comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
The scene recognition system, method and device based on the intelligent gateway, the intelligent gateway and the storage medium comprise: the system comprises an intelligent gateway and camera equipment; the intelligent gateway is in communication connection with the camera equipment; the camera shooting device is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified; the intelligent gateway is used for acquiring a scene image to be identified from the camera equipment; inputting a scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier; and sending the video stream data to a corresponding anomaly detection server according to the scene category identification. The intelligent gateway identifies the scene image to be identified in the video stream data acquired by the camera equipment to obtain a scene identification result; and the video stream data is sent to the corresponding anomaly detection server according to the scene category identification in the identification result, so that the identification, classification and sending processing of the video stream data are realized, and the scene identification efficiency is improved.
Drawings
FIG. 1 is a schematic structural diagram of an intelligent gateway based scene recognition system in one embodiment;
FIG. 2 is a diagram of an application environment of the intelligent gateway based scene recognition method in one embodiment;
FIG. 3 is a schematic flowchart of a scene recognition method based on an intelligent gateway in an embodiment;
FIG. 4 is a schematic flow chart of the pre-constructed scene recognition model acquisition step in one embodiment;
FIG. 5 is a block diagram of an intelligent gateway based scene recognition apparatus according to an embodiment;
fig. 6 is an internal structural diagram of an intelligent gateway in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The structure of the scene recognition system based on the intelligent gateway is shown in fig. 1, and the system comprises 11 intelligent gateways and 12 camera devices; the intelligent gateway 11 is in communication connection with the camera device 12;
the camera device 12 is configured to acquire video stream data of a scene to be identified, where the video stream data includes multiple frames of scene images to be identified;
the intelligent gateway 11 is used for acquiring a scene image to be identified from the camera device 12; inputting a scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier; and the intelligent gateway 11 sends the video stream data to a corresponding anomaly detection server according to the scene category identification.
The camera device can acquire video stream data of a scene to be identified in real time, and the video stream data can reflect the current real-time condition of the scene to be identified.
The scene to be recognized can be a preset area, can also be a specific power device, and can also be an instrument and a data display screen of the instrument; through obtaining the data display screen of the instrument, corresponding operation data can be obtained without sending personnel to a field.
The video stream data is composed of a plurality of frames of images on a data structure, and the number of the image frames is related to the number of frames of the video stream; one frame is a static picture, and the continuous frames form a video stream; therefore, a plurality of frames of scene images to be identified can be acquired through the video stream data.
The intelligent gateways are core modules realized by an industrial personal computer (industrial personal computer) or an embedded processor, generally have access capability of a part of hardware interfaces and conversion capability of a part of hardware interface protocols, can be connected with a plurality of types of equipment at the same time, can realize conversion of the interface protocols, and improve the convenience of mutual compatibility and mutual communication among the equipment. The intelligent gateway can realize functions of data processing, equipment control and the like to a certain degree on the basis of a conventional gateway, and is a novel gateway component; the intelligent gateway in the disclosure can realize the functions of acquiring, storing and sending video stream data, and further can recognize the monitoring scene corresponding to the video stream data through the scene image to be recognized in the video stream data so as to realize the distribution of the video stream data.
The pre-constructed scene recognition model is a neural network model suitable for the data processing capacity of the intelligent gateway, and can recognize scenes corresponding to transmitted video stream data on the basis of not influencing the basic data receiving and forwarding functions of the intelligent gateway. The pre-constructed scene recognition model can be directly constructed and trained on the intelligent gateway, and the trained neural network model on other equipment can be transplanted to the intelligent gateway for application.
The scene recognition result refers to a prediction result output after a pre-constructed scene recognition model processes an input scene image to be recognized; the scene identification result contains a scene type identifier, and the scene corresponding to the video stream data can be judged through the scene type identifier.
The scene category identification can be set in groups in advance, for example, the group A is various meter devices including a winding thermometer, an oil surface thermometer, an oil level meter and the like; the group B is set to be various electrical equipment, including a respirator, a lightning arrester, a detector for the lightning arrester, a switch control cabinet and the like; the group C is set to various knife switch devices, including a single-arm vertical knife switch, a seven-type knife switch, a four-type knife switch and the like.
The abnormality detection server is a server capable of identifying and detecting scenes in the received video stream data; for example, the intelligent gateway sends video stream data of the meter devices in the group a to the abnormality detection server a1, so that the server a1 can identify the display degrees of the meter devices, judge that the indexes monitored by the meter devices are in an abnormal state when the degrees exceed a preset value, and remind corresponding personnel to go to the site for processing. For another example, the intelligent gateway sends video stream data of the electrical devices in the group B to the abnormality detection server B1, so that the server B1 can identify the defect condition of the electrical devices, for example, if a crack is identified in the respirator, corresponding personnel is reminded to go to the site to process the crack.
Specifically, the camera device collects video stream data or image data of a scene to be identified in real time or periodically, and transmits the collected data to the intelligent gateway through a network; after receiving video stream data, the intelligent gateway extracts an image capable of representing the video stream data from the video stream data as a scene image to be identified; the method comprises the steps of inputting a scene image to be recognized into a pre-constructed scene recognition model arranged in an intelligent gateway, obtaining a corresponding scene type identifier according to a scene recognition result output by the pre-constructed scene recognition model, determining a corresponding abnormity detection server according to the scene type identifier, and sending video stream data or corresponding image data to the abnormity detection server.
The intelligent gateway identifies the scene image to be identified in the video stream data acquired by the camera equipment to obtain a scene identification result; and the video stream data is sent to the corresponding anomaly detection server according to the scene category identification in the identification result, so that the identification, classification and sending processing of the video stream data are realized, and the scene identification efficiency is improved.
In one embodiment, the intelligent gateway is further configured to control the camera device to rotate in response to an instruction of the terminal device, so that the camera device collects video stream data of different scenes to be identified.
The terminal device is a device capable of sending an instruction to the intelligent gateway, such as a mobile phone, a computer, a portable wearable device, and the like of a power equipment maintenance worker.
The camera shooting device can rotate to acquire video stream data of different scenes in the environment.
Specifically, the terminal device sends an instruction to the intelligent gateway, the intelligent gateway analyzes the instruction, and determines the identifier of the camera device to be rotated, the specific rotation angle, the time for returning to the original state after rotation, and other parameters according to the analysis result, and controls the camera device to rotate. According to the embodiment, the intelligent gateway controls the camera shooting equipment to rotate according to the instruction, so that the camera shooting equipment can collect video stream data of a plurality of scenes, the acquisition efficiency of the video stream data is improved, and the installation cost of the camera shooting equipment is reduced.
In one embodiment, the system further comprises: a database; the database is in communication connection with the intelligent gateway and is used for receiving video stream data sent to the abnormality detection server by the intelligent gateway and storing the video stream data.
Specifically, the database can store video stream data transmitted by the intelligent gateway, and can encrypt the video stream data during storage, so that the safety of the video stream data is improved.
In one embodiment, the anomaly detection server is further configured to obtain video stream data in a database; carrying out anomaly detection on a scene image in video stream data; and if the scene image is detected to be abnormal, generating alarm information and sending the alarm information to a preset terminal.
The alarm information refers to warning information capable of reminding people. The anomaly detection refers to a process of determining whether information such as power equipment, environment, meter index and the like in a scene image has anomalies. The preset terminal is an object device for receiving the alarm information, for example, a mobile phone, a computer, etc. of a maintenance person can be used as the preset terminal for receiving the alarm information.
In one embodiment, the pre-constructed scene recognition model is obtained by training a scene recognition model to be trained, which is constructed based on a YOLO algorithm, for multiple times; the scene recognition model to be trained comprises a convolution layer, a pooling layer and a full-connection layer.
The YOLO algorithm is a high-performance target detection framework based on a deep learning network, is realized by adopting C language, and can accurately detect scenes in a monitoring picture scene; objects are located and marked by drawing a border around the object and identifying class labels to which a given frame also belongs. Unlike a large NLP (Natural Language Processing) algorithm, the YOLO algorithm is designed to be small and can be deployed on various computationally-powered devices to provide real-time reasoning and improve data Processing speed.
Wherein, the Convolutional layer (Convolutional layer) is composed of a plurality of convolution units, and the parameter of each convolution unit is optimized by a back propagation algorithm. The convolution operation aims to extract different input features, the convolution layer at the first layer can only extract some low-level features such as edges, lines, angles and other levels, and more layers of networks can iteratively extract more complex features from the low-level features. The pooling layer is sandwiched between successive convolutional layers and serves to compress the amount of data and parameters, reducing overfitting, the effect being interpreted as compressing the image. Each node of the fully connected layer is connected to all nodes of the previous layer for integrating the extracted features.
The scene recognition method based on the intelligent gateway can be applied to the application environment shown in fig. 2. The imaging device 21 and the abnormality detection server 22 communicate with the smart gateway 23 via a network. The intelligent gateway 23 acquires a scene image to be identified from the camera device 21; the camera device 21 is configured to collect video stream data of a scene to be identified, where the video stream data includes multiple frames of scene images to be identified; the intelligent gateway 23 inputs the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier; the intelligent gateway 23 sends the video stream data to the corresponding anomaly detection server 22 according to the scene category identification.
In one embodiment, as shown in fig. 3, a scene recognition method based on an intelligent gateway is provided, which is described by taking the method as an example of being applied to the intelligent gateway 23 in fig. 1, and includes the following steps:
And step 32, inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing the scene category identification.
And step 33, sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
Specifically, the camera device collects video stream data or image data of a scene to be identified in real time or periodically, and transmits the collected data to the intelligent gateway through a network; after receiving video stream data, the intelligent gateway extracts an image capable of representing the video stream data from the video stream data as a scene image to be identified; the method comprises the steps of inputting a scene image to be recognized into a pre-constructed scene recognition model arranged in an intelligent gateway, obtaining a corresponding scene type identifier according to a scene recognition result output by the pre-constructed scene recognition model, determining a corresponding abnormity detection server according to the scene type identifier, and sending video stream data or corresponding image data to the abnormity detection server.
In the scene identification method based on the intelligent gateway, the intelligent gateway identifies the scene image to be identified in the video stream data acquired by the camera equipment to obtain a scene identification result; and the video stream data is sent to the corresponding anomaly detection server according to the scene category identification in the identification result, so that the identification, classification and sending processing of the video stream data are realized, and the scene identification efficiency is improved.
In one embodiment, as shown in fig. 4, in the step 22, the pre-constructed scene recognition model is obtained by the following steps:
and 43, adjusting model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as a pre-constructed scene recognition model.
The sample image set is a data set composed of sample images for model training. The scene type refers to the type of the corresponding device or area in the picture content of the sample image; for example, a knife switch and a switch cabinet are respectively two scene categories. The loss value (loss) is obtained from a loss function, which is a function representing the "risk" or "loss" of an event. In the present disclosure, the loss value is obtained according to the difference between the recognition result of the sample image and the real result of the sample image according to the scene recognition model to be trained. The parameter refers to a variable parameter inside the scene recognition model to be trained, and for the neural network model, the parameter may also be referred to as a neural network weight (weight). Model parameters in the scene recognition model to be trained can be adjusted towards the direction that the loss value becomes smaller, and the pre-constructed scene recognition model is obtained through multiple times of iterative training.
In the embodiment, the sample image is used for carrying out iterative training on the scene recognition model to be trained for multiple times, so that the trained scene recognition model can accurately recognize scenes in the sample image, the scene recognition model is used for practice, the functions of recognizing and classifying video stream data are realized, and the scene recognition efficiency is improved.
It should be understood that although the various steps in the flow charts of fig. 3-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 3-4 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 5, there is provided a scene recognition apparatus based on an intelligent gateway, including: an image acquisition module 51, a scene recognition module 52 and a data transmission module 53, wherein:
an image acquisition module 51, configured to acquire a scene image to be identified from an image capturing apparatus; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified;
the scene recognition module 52 is configured to input a to-be-recognized scene image into a pre-constructed scene recognition model, and obtain a scene recognition result corresponding to the to-be-recognized scene image;
and a data sending module 53, configured to send the video stream data to the corresponding terminal device according to the scene identification result.
In one embodiment, the intelligent gateway-based scene recognition device further comprises a model training module for obtaining a sample image set; the sample image set comprises sample images of a plurality of scene categories; inputting the sample image set into a scene recognition model to be trained, performing iterative training on the scene recognition model to be trained by adopting a loss function, and obtaining a loss value according to the class probability corresponding to each scene class; and adjusting model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as a pre-constructed scene recognition model.
For specific limitations of the intelligent gateway based scene recognition apparatus, reference may be made to the above limitations of the intelligent gateway based scene recognition method, which is not described herein again. The modules in the intelligent gateway-based scene recognition device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the intelligent gateway, and can also be stored in a memory in the intelligent gateway in a software form, so that the processor can call and execute the corresponding operations of the modules.
In one embodiment, an intelligent gateway is provided, which may be a terminal, and the internal structure diagram thereof may be as shown in fig. 6. The intelligent gateway comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the intelligent gateway is configured to provide computing and control capabilities. The memory of the intelligent gateway comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the intelligent gateway is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an intelligent gateway based scene recognition method. The display screen of the intelligent gateway can be a liquid crystal display screen or an electronic ink display screen, and the input device of the intelligent gateway can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the intelligent gateway, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is a block diagram of only a portion of the architecture associated with the subject application, and does not constitute a limitation on the intelligent gateway to which the subject application applies, and that a particular intelligent gateway may include more or fewer components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, an intelligent gateway is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring a scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified;
inputting a scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring a sample image set; the sample image set comprises sample images of a plurality of scene categories; inputting the sample image set into a scene recognition model to be trained, performing iterative training on the scene recognition model to be trained by adopting a loss function, and obtaining a loss value according to the class probability corresponding to each scene class; and adjusting model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as a pre-constructed scene recognition model.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified;
inputting a scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a sample image set; the sample image set comprises sample images of a plurality of scene categories; inputting the sample image set into a scene recognition model to be trained, performing iterative training on the scene recognition model to be trained by adopting a loss function, and obtaining a loss value according to the class probability corresponding to each scene class; and adjusting model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as a pre-constructed scene recognition model.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A scene recognition system based on an intelligent gateway is characterized by comprising: the system comprises an intelligent gateway and camera equipment; the intelligent gateway is in communication connection with the camera equipment;
the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of scene images to be identified;
the intelligent gateway is used for acquiring the scene image to be identified from the camera equipment; inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier; and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
2. The system according to claim 1, wherein the intelligent gateway is further configured to control the camera device to rotate in response to an instruction of the terminal device, so that the camera device collects video stream data of different scenes to be recognized.
3. The system of claim 1, further comprising: a database;
the database is in communication connection with the intelligent gateway and is used for receiving video stream data sent to the anomaly detection server by the intelligent gateway and storing the video stream data.
4. The system of claim 3, wherein the anomaly detection server is further configured to obtain video stream data in the database; carrying out anomaly detection on a scene image in the video stream data; and if the scene image is detected to be abnormal, generating alarm information and sending the alarm information to a preset terminal.
5. The system according to any one of claims 1-4, wherein the pre-constructed scene recognition model is obtained by training a scene recognition model to be trained, which is constructed based on a YOLO algorithm, for a plurality of times; the scene recognition model to be trained comprises a convolution layer, a pooling layer and a full-connection layer.
6. A scene recognition method based on an intelligent gateway is characterized by comprising the following steps:
acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result containing a scene category identifier;
and sending the video stream data to a corresponding anomaly detection server according to the scene category identification.
7. The method according to claim 6, wherein the pre-constructed scene recognition model is obtained by:
acquiring a sample image set; the sample image set comprises sample images of a plurality of scene categories;
inputting the sample image set into a scene recognition model to be trained, performing iterative training on the scene recognition model to be trained by adopting a loss function, and obtaining a loss value according to the class probability corresponding to each scene class;
and adjusting the model parameters of the scene recognition model to be trained according to the loss value until the loss value of the adjusted scene recognition model to be trained is lower than a preset threshold value, and taking the adjusted scene recognition model to be trained as the pre-constructed scene recognition model.
8. An intelligent gateway based scene recognition device, the device comprising:
the image acquisition module is used for acquiring the scene image to be identified from the camera equipment; the camera shooting equipment is used for collecting video stream data of a scene to be identified, and the video stream data comprises a plurality of frames of images of the scene to be identified;
the scene recognition module is used for inputting the scene image to be recognized into a pre-constructed scene recognition model to obtain a scene recognition result corresponding to the scene image to be recognized;
and the data sending module is used for sending the video stream data to the corresponding terminal equipment according to the scene identification result.
9. An intelligent gateway comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any one of claims 6 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110839664.7A CN113723184A (en) | 2021-07-23 | 2021-07-23 | Scene recognition system, method and device based on intelligent gateway and intelligent gateway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110839664.7A CN113723184A (en) | 2021-07-23 | 2021-07-23 | Scene recognition system, method and device based on intelligent gateway and intelligent gateway |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113723184A true CN113723184A (en) | 2021-11-30 |
Family
ID=78673933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110839664.7A Pending CN113723184A (en) | 2021-07-23 | 2021-07-23 | Scene recognition system, method and device based on intelligent gateway and intelligent gateway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723184A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114338284A (en) * | 2021-12-24 | 2022-04-12 | 深圳尊悦智能科技有限公司 | 5G intelligent gateway of Internet of things |
| CN115690897A (en) * | 2022-08-31 | 2023-02-03 | 北京夕阳无忧科技有限公司 | Accidental behavior processing method, device, equipment and storage medium for preventing privacy leakage |
| CN116645530A (en) * | 2023-04-23 | 2023-08-25 | 广东建瀚工程管理有限公司 | Construction detection method, device, equipment and storage medium based on image comparison |
-
2021
- 2021-07-23 CN CN202110839664.7A patent/CN113723184A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114338284A (en) * | 2021-12-24 | 2022-04-12 | 深圳尊悦智能科技有限公司 | 5G intelligent gateway of Internet of things |
| CN115690897A (en) * | 2022-08-31 | 2023-02-03 | 北京夕阳无忧科技有限公司 | Accidental behavior processing method, device, equipment and storage medium for preventing privacy leakage |
| CN115690897B (en) * | 2022-08-31 | 2023-10-20 | 北京夕阳无忧科技有限公司 | Unexpected behavior processing method, device and equipment for preventing privacy leakage and storage medium |
| CN116645530A (en) * | 2023-04-23 | 2023-08-25 | 广东建瀚工程管理有限公司 | Construction detection method, device, equipment and storage medium based on image comparison |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109980781B (en) | Intelligent monitoring system of transformer substation | |
| CN113723184A (en) | Scene recognition system, method and device based on intelligent gateway and intelligent gateway | |
| US10812761B2 (en) | Complex hardware-based system for video surveillance tracking | |
| CN107908175A (en) | A kind of electric system site intelligent operational system | |
| CN108090458A (en) | Tumble detection method for human body and device | |
| CN104167818B (en) | Intelligent remote inspection system in linkage with GIS substation integrated automation system and method | |
| WO2022160413A1 (en) | Electric power production anomaly monitoring method and apparatus, and computer device and storage medium | |
| CN111918039B (en) | Artificial intelligence high risk operation management and control system based on 5G network | |
| CN110110869A (en) | A kind of power station intelligent inspection system | |
| CN109974780A (en) | A kind of electrical equipment status monitoring system based on Internet of Things | |
| CN114665608B (en) | Intelligent sensing inspection system and method for transformer substation | |
| CN112688434A (en) | Monitoring and early warning method and device for power transmission and distribution line, computer equipment and medium | |
| KR20220055082A (en) | System and method for defect detection based on deep learning through machine-learning of solar module data of thermal image | |
| CN115103157A (en) | Video analysis method and device based on edge cloud cooperation, electronic equipment and medium | |
| CN110505438B (en) | Queuing data acquisition method and camera | |
| CN113628172A (en) | Intelligent detection algorithm for personnel handheld weapons and smart city security system | |
| CN112016380A (en) | Wild animal monitoring method and system | |
| CN108073854A (en) | A kind of detection method and device of scene inspection | |
| CN116744357A (en) | Base station fault prediction method, device, equipment and medium | |
| CN111860187A (en) | High-precision worn mask identification method and system | |
| CN114387391A (en) | Safety monitoring method and device for transformer substation equipment, computer equipment and medium | |
| CN111291597B (en) | Crowd situation analysis method, device, equipment and system based on image | |
| CN113449015A (en) | Power grid fault processing method and device and electronic equipment | |
| Gao et al. | Power transmission line anomaly detection scheme based on CNN-transformer model | |
| CN111859370A (en) | Method, apparatus, electronic device and computer-readable storage medium for identifying service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |