CN115147715A - Fire detection method and device based on TinyML - Google Patents

Fire detection method and device based on TinyML Download PDF

Info

Publication number
CN115147715A
CN115147715A CN202210387824.3A CN202210387824A CN115147715A CN 115147715 A CN115147715 A CN 115147715A CN 202210387824 A CN202210387824 A CN 202210387824A CN 115147715 A CN115147715 A CN 115147715A
Authority
CN
China
Prior art keywords
fire detection
tinyml
detection model
model
mobilenet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210387824.3A
Other languages
Chinese (zh)
Inventor
段强
李锐
张晖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Inspur Science Research Institute Co Ltd
Original Assignee
Shandong Inspur Science Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Inspur Science Research Institute Co Ltd filed Critical Shandong Inspur Science Research Institute Co Ltd
Priority to CN202210387824.3A priority Critical patent/CN115147715A/en
Publication of CN115147715A publication Critical patent/CN115147715A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Fire-Detection Mechanisms (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a fire detection method and a fire detection device based on TinyML, and relates to the technical field of machine learning application; the method comprises the steps of disassembling based on an MCU (microprogrammed control Unit) microprocessor, establishing a MobileNet-v1 neural network of a TinyML algorithm step by using a C language, carrying out fire detection model training by using a public data set according to the MobileNet-v1 neural network, extracting and quantifying the weight of the fire detection model, putting the quantified weight of the fire detection model into a code of the MobileNet-v1 neural network as a hyper-parameter, reasoning whether flame exists in an application scene image acquired from a cloud end through the fire detection model, and awakening an edge end device to acquire the image to accurately position the flame if the flame exists.

Description

Fire detection method and device based on TinyML
Technical Field
The invention discloses a method and a device, relates to the technical field of machine learning application, and particularly relates to a fire detection method and a fire detection device based on TinyML.
Background
At present, fire detection usually has two schemes, one scheme is to extract image videos through a camera and transmit the images to a cloud end for cloud service detection and early warning, and the other scheme is to directly detect through directly integrating an algorithm model and the like in the camera and only return a result for early warning monitoring. The first scheme has low requirements on equipment of a field implementation part, only needs to have the functions of shooting, recording and network transmission, has high requirements on the network transmission part, needs to be transmitted to a cloud in real time and needs to be stably connected, and has high cost and power consumption. The second scheme has a high requirement on integration level, and needs to integrate intelligent sensing and computing units in a limited internal space of a camera, and the intelligent sensing and computing units are usually edge computing units and machine vision algorithms based on an ARM architecture, so that the high integration level is relatively high in cost, and the algorithm running at full speed continuously consumes relatively high power.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a fire detection method and a fire detection device based on TinyML, wherein a TinyML algorithm and a camera are combined with an MCU (microprogrammed control Unit) to be used as a visual awakening device, a scene fire is detected by continuous low-power-consumption operation, and when the fire is detected to exist, a signal is sent to awaken a high-definition camera and a network device for subsequent disposal.
The specific scheme provided by the invention is as follows:
the invention provides a fire detection method based on TinyML, which is based on MCU (microprogrammed control Unit) microprocessor disassembly and step-by-step establishment of a MobileNet-v1 neural network of a TinyML algorithm by C language, carries out fire detection model training by utilizing a public data set according to the MobileNet-v1 neural network, extracts and quantifies the weight of the fire detection model, takes the weight of the quantified fire detection model as a hyper-parameter and puts the hyper-parameter into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
Further, in the fire detection method based on tinyML, the MobileNet-v1 neural network based on the tinyML algorithm is disassembled based on the MCU microprocessor and is established step by using the C language, and the method comprises the following steps:
and establishing a MobileNet-v1 neural network layer which comprises a depth separable convolutional layer and a full connection layer.
Further, in the fire detection method based on TinyML, the training of the fire detection model by using the public data set according to the MobileNet-v1 neural network includes:
using ImageNet-1k data set to train the fire detection model to obtain a pre-training model,
and collecting data according to specific tasks and performing transfer learning of the pre-training model to obtain a fire detection model.
Further, in the method for detecting a fire based on TinyML, the extracting and quantizing the weight of the fire detection model includes:
according to different training frames of the fire detection model, selecting a corresponding mode to extract the weight of the fire detection model, carrying out pre-quantization on the weight,
using a quantization formula x quantized And (= 255) ((x _ float-x _ min)/(x _ max-x _ min)), quantizing the pre-quantized weights, wherein x _ float is an original value of each datum of the model weights, x _ min is a minimum value of all values of the model weights, and x _ max is a maximum value of all values of the model weights.
Further, in the fire detection method based on tinyML, the MCU-based microprocessor comprises:
an MCU microprocessor based on RISC-V architecture and using an FPGA-based convolution acceleration unit.
The invention also provides a fire detection device based on TinyML, which comprises an MCU module,
the MCU module is disassembled based on the MCU microprocessor, a MobileNet-v1 neural network of a TinyML algorithm is built step by using C language, a fire detection model is trained by utilizing a public data set according to the MobileNet-v1 neural network, the weight of the fire detection model is extracted and quantized, the quantized weight of the fire detection model is used as a hyper-parameter and is put into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
Further, the MobileNet-v1 neural network for the disassembly of the MCU module and the stepwise establishment of the TinyML algorithm by the C language in the fire detection device based on the TinyML comprises:
and establishing a MobileNet-v1 neural network layer comprising a deep separable convolutional layer and a full connection layer.
Further, in the fire detection device based on tinyML, the MCU module performs fire detection model training by using a public data set according to a MobileNet-v1 neural network, and the method comprises the following steps:
using ImageNet-1k data set to train the fire detection model to obtain a pre-training model,
and collecting data according to specific tasks and performing transfer learning of the pre-training model to obtain a fire detection model.
Further, in the fire detection device based on TinyML, the MCU module extracts and quantizes the weight of the fire detection model, including:
according to different training frames of the fire detection model, selecting a corresponding mode to extract the weight of the fire detection model, carrying out pre-quantization on the weight,
using a quantization formula x quantized And (= 255) ((x _ float-x _ min)/(x _ max-x _ min)), quantizing the pre-quantized weights, wherein x _ float is an original value of each datum of the model weights, x _ min is a minimum value of all values of the model weights, and x _ max is a maximum value of all values of the model weights.
Further, in the fire detection device based on tinyML, the MCU module is an MCU microprocessor based on RISC-V architecture, and an FPGA-based convolution acceleration unit is used.
The invention has the advantages that:
the invention provides a fire detection method based on TinyML, which detects scene flames through continuous low-power-consumption operation by taking an MCU microprocessor which operates continuously with low power consumption and a TinyML algorithm as a sentinel, wakes up an intelligent camera of edge equipment only when needed, is very beneficial to power consumption control, and is beneficial to subsequent treatment such as alarming, field picture transmission and the like.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the embodiments or technical solutions in the prior art are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of the process of the present invention.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides a fire detection method based on TinyML, which is based on MCU (microprogrammed control Unit) microprocessor disassembly and step-by-step establishment of a MobileNet-v1 neural network of a TinyML algorithm by C language, carries out fire detection model training by utilizing a public data set according to the MobileNet-v1 neural network, extracts and quantifies the weight of the fire detection model, takes the weight of the quantified fire detection model as a hyper-parameter and puts the hyper-parameter into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
The network structure of the MobileNet-v1 of the TinyML algorithm is established through the MCU, the scene flame is detected through continuous low-power-consumption operation, and when the flame is detected to exist, a signal is sent to wake up the high-definition camera and the network equipment for subsequent treatment, such as secondary confirmation, alarming, field picture transmission and the like.
In specific application, in some embodiments of the method, the MCU is utilized to disassemble and establish the MobileNet-v1 neural network of the TinyML algorithm step by using C language during fire detection based on the TinyML.
Further, as for the MCU microprocessor, a RISC-V architecture microprocessor is used, and an FPGA-based convolution acceleration unit may be used as necessary. The RISC-V has the characteristics of open source, light weight, modular design and the like, and has remarkable advantages. In order to improve the calculation efficiency and energy efficiency of a Convolutional Neural Network (CNN), 8-bit integer data is used as input, a convolutional accelerator supporting common calculation types in the CNN network such as activation, batch normalization and pooling is used, the cyclic calculation sequence is optimized, and the cyclic calculation sequence is combined with a data multiplexing technology to improve the efficiency of convolutional calculation. Based on the software and hardware collaborative design idea, an SoC system comprising a RISC-V processor and a convolution accelerator is constructed, and the RISC-V processor can expand instruction functions according to specific design requirements based on the open source instruction set standard.
Further, for the MobileNet-v1 neural network for establishing the TinyML algorithm, wherein MobileNet v1 may include sixteen neural network layers with trainable weights, including 13 depth separable convolutional layers and 3 fully-connected layers, and the depth separable convolutional layers may each be a combination of depth convolution and point-by-point convolution, so that the operation of achieving low parameter quantity performs approximate calculation on 2D convolution. The images in the camera view field are classified through the network, and the classification is fire or no fire. Further realize the batch normalization layer, the pooling layer and the ReLU layer.
Further, when the fire detection model is trained by using the public data set according to the MobileNet-v1 neural network, the ImageNet-1k data set can be used for model training to obtain a pre-training model, and then data is collected according to specific tasks and is subjected to transfer learning.
And further, according to different frames of model training, selecting a corresponding mode to extract the model weight, and carrying out pre-quantization on the weight by using fixed-point quantization on the extracted array/matrix, such as adding Post-training quantization, namely converting the numerical value in the original network weight from Float32 to int8, so that the range of the quantized weight is [0,255]. The quantitative model is:
x quantized =255*(x_float-x_min)/(x_max-x_min)
where x _ float is the original value of each data of the model weights, x _ min is the minimum of all values of the model weights, x _ max is the maximum of all values of the model weights,
the model outputs the classification result for the captured image, and the confidence is obtained after softmax. And setting a confidence threshold h (usually 0.5), sending a wake-up signal when the confidence exceeds h, and transmitting the wake-up signal to the cloud end by using an intelligent monitor or a wake-up network transmission device to perform further accurate judgment as a final result.
The quantized weight is put into the built neural network code as a hyper-parameter, namely the weight is directly used as a static array to be directly initialized in the C code without reading binary or text files,
and MCU microprocessing software and hardware are deployed in a real scene, the MCU microprocessing software and hardware are linked with edge equipment through LAN lines or GSM signals are linked with a cloud end, whether flames exist in application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge equipment is awakened to acquire images through LAN or GSM signals to accurately position the flames.
The invention also provides a fire detection device based on TinyML, which comprises an MCU module,
the MCU module is disassembled based on the MCU microprocessor, a MobileNet-v1 neural network of a TinyML algorithm is built step by using C language, a fire detection model is trained by utilizing a public data set according to the MobileNet-v1 neural network, the weight of the fire detection model is extracted and quantized, the quantized weight of the fire detection model is used as a hyper-parameter and is put into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
Because the content of information interaction, execution process, and the like among the modules in the device is based on the same concept as the method embodiment of the present invention, specific content can be referred to the description in the method embodiment of the present invention, and is not described herein again.
Similarly, the device can be used as a sentinel by using the MCU microprocessor and the TinyML algorithm which run continuously with low power consumption, detect scene flames through continuous low power consumption running, wake up the intelligent camera of the edge device only when needed, is very beneficial to power consumption control, and is beneficial to subsequent treatment such as alarming, field picture transmission and the like.
It should be noted that not all steps and modules in the above flows and device structures are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.

Claims (10)

1. A fire detection method based on TinyML is characterized in that a MobileNet-v1 neural network of a TinyML algorithm is disassembled based on an MCU (microprogrammed control Unit) microprocessor and built step by using C language, a fire detection model is trained by utilizing a public data set according to the MobileNet-v1 neural network, the weight of the fire detection model is extracted and quantized, the weight of the quantized fire detection model is taken as a hyper-parameter and is put into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
2. The method as claimed in claim 1, wherein the method for detecting fire based on tinyML comprises the following steps:
and establishing a MobileNet-v1 neural network layer which comprises a depth separable convolutional layer and a full connection layer.
3. The method for fire detection based on TinyML according to claim 1 or 2, wherein the training of the fire detection model using the public data set according to the MobileNet-v1 neural network comprises:
using ImageNet-1k data set to train the fire detection model to obtain a pre-training model,
and collecting data according to specific tasks and performing transfer learning of the pre-training model to obtain a fire detection model.
4. The method for fire detection based on TinyML as claimed in claim 1, wherein the extracting and quantizing the weight of the fire detection model comprises:
according to different training frames of the fire detection model, selecting a corresponding mode to extract the weight of the fire detection model, carrying out pre-quantization on the weight,
using a quantization formula x quantized And =255 = (x _ float-x _ min)/(x _ max-x _ min), where x _ float is the original value of each data of the model weight, x _ min is the minimum value of all values of the model weight, and x _ max is the maximum value of all values of the model weight.
5. The TinyML-based fire detection method as recited in claim 1, wherein the MCU microprocessor-based microprocessor comprises:
an MCU microprocessor based on RISC-V architecture and using a convolution acceleration unit based on FPGA.
6. A fire detection device based on TinyML is characterized by comprising an MCU module,
the MCU module is disassembled based on the MCU microprocessor, a MobileNet-v1 neural network of a TinyML algorithm is built step by using C language, a fire detection model is trained by utilizing a public data set according to the MobileNet-v1 neural network, the weight of the fire detection model is extracted and quantized, the quantized weight of the fire detection model is used as a hyper-parameter and is put into a code of the MobileNet-v1 neural network,
whether flames exist in the application scene images acquired from the cloud end or not is inferred through the fire detection model, and if the flames exist, the edge end equipment is awakened to acquire the images to accurately position the flames.
7. The fire detection device based on tinyML as claimed in claim 6, wherein the MCU module disassembles and builds the MobileNet-v1 neural network of the tinyML algorithm step by step with the C language, comprising:
and establishing a MobileNet-v1 neural network layer which comprises a depth separable convolutional layer and a full connection layer.
8. The TinyML-based fire detection device as recited in claim 6 or 7, wherein the MCU module performs fire detection model training using public data sets according to MobileNet-v1 neural network, comprising:
using ImageNet-1k data set to train the fire detection model to obtain a pre-training model,
and collecting data according to specific tasks and performing transfer learning of the pre-training model to obtain a fire detection model.
9. The TinyML-based fire detection device as claimed in claim 6, wherein the MCU module extracts and quantifies the weight of the fire detection model, and comprises:
according to different training frames of the fire detection model, selecting a corresponding mode to extract the weight of the fire detection model, carrying out pre-quantization on the weight,
using a quantization formula x quantized And (= 255) ((x _ float-x _ min)/(x _ max-x _ min)), quantizing the pre-quantized weights, wherein x _ float is an original value of each datum of the model weights, x _ min is a minimum value of all values of the model weights, and x _ max is a maximum value of all values of the model weights.
10. The fire detection device based on TinyML as claimed in claim 6, wherein the MCU module is an MCU microprocessor based on RISC-V architecture and uses an FPGA based convolution acceleration unit.
CN202210387824.3A 2022-04-14 2022-04-14 Fire detection method and device based on TinyML Pending CN115147715A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210387824.3A CN115147715A (en) 2022-04-14 2022-04-14 Fire detection method and device based on TinyML

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210387824.3A CN115147715A (en) 2022-04-14 2022-04-14 Fire detection method and device based on TinyML

Publications (1)

Publication Number Publication Date
CN115147715A true CN115147715A (en) 2022-10-04

Family

ID=83406430

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210387824.3A Pending CN115147715A (en) 2022-04-14 2022-04-14 Fire detection method and device based on TinyML

Country Status (1)

Country Link
CN (1) CN115147715A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112699963A (en) * 2021-01-13 2021-04-23 四川九通智路科技有限公司 Fire detection method
CN113033553A (en) * 2021-03-22 2021-06-25 深圳市安软科技股份有限公司 Fire detection method and device based on multi-mode fusion, related equipment and storage medium
CN113627493A (en) * 2021-07-23 2021-11-09 四川九通智路科技有限公司 Fire detection method based on convolutional neural network model
CN114298167A (en) * 2021-12-10 2022-04-08 西安航空学院 Tunnel fire detection method based on YOLO neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112699963A (en) * 2021-01-13 2021-04-23 四川九通智路科技有限公司 Fire detection method
CN113033553A (en) * 2021-03-22 2021-06-25 深圳市安软科技股份有限公司 Fire detection method and device based on multi-mode fusion, related equipment and storage medium
CN113627493A (en) * 2021-07-23 2021-11-09 四川九通智路科技有限公司 Fire detection method based on convolutional neural network model
CN114298167A (en) * 2021-12-10 2022-04-08 西安航空学院 Tunnel fire detection method based on YOLO neural network

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
MJROBOT(MARCELO ROVAL): "ESP32-CAM:TinyML image Classification-Fruits vs Veggies", Retrieved from the Internet <URL:https://www.hackster.io/ mjrobot/esp32-cam-tinyml-image-classification-fruits-vs-veggies-4ab970#things> *
严泰来等: "《遥感技术与农业应用》", 31 July 2008, 中国农业大学出版社 *
周坚: "《单片机项目教程-C语言版》", 31 March 2013, 北京航空航天大学出版社 *
彭昭: "《数字经济与5G新商机》", 31 July 2021, 北京理工大学出版社 *
杨聪等: "《智能座舱开发与实践》", 28 February 2022, 机械工业出版社 *
樊继明等: "《FPGA深度解析》", 31 May 2015, 北京航空航天大学出版社 *

Similar Documents

Publication Publication Date Title
CN110163143A (en) Unlawful practice recognition methods, device and terminal device
CN105447473A (en) PCANet-CNN-based arbitrary attitude facial expression recognition method
CN112087442B (en) Time sequence related network intrusion detection method based on attention mechanism
CN108805008A (en) A kind of community&#39;s vehicle security system based on deep learning
CN109101108A (en) Method and system based on three decision optimization intelligence cockpit human-computer interaction interfaces
CN109259528A (en) A kind of home furnishings intelligent mirror based on recognition of face and skin quality detection
CN112487877A (en) Monitoring method, system, device and medium for standard operation of kitchen waste
CN115147715A (en) Fire detection method and device based on TinyML
CN117292330B (en) Intelligent monitoring system suitable for time sequence data operation and maintenance
CN111126155A (en) Pedestrian re-identification method for generating confrontation network based on semantic constraint
Wang et al. Spiking emotions: Dynamic vision emotion recognition using spiking neural networks
CN116631566A (en) Medical image report intelligent generation method based on big data
CN116188870A (en) Steel surface defect image classification method based on pulse convolution neural network
CN115512180A (en) Low-power consumption intelligent monitoring management system
CN115205786A (en) On-line automatic identification and alarm method for mobile phone pirate behavior
CN114118129A (en) Method for detecting urban lighting facilities
CN109951866B (en) People flow monitoring method based on hidden Markov model
CN209993011U (en) Low-power-consumption online thermal infrared target recognition device
Chen et al. A multi-view driver drowsiness detection method using transfer learning and population-based sampling strategy
Wu et al. Flame and smoke detection algorithm for UAV based on improved YOLOv4-Tiny
CN113780439B (en) Multi-cloud identification system of different types of meteorological satellites based on unsupervised domain adaptation
CN112541081A (en) Migratory rumor detection method based on field self-adaptation
CN117315428B (en) Cross-modal feature alignment and fusion crowd counting system and method
CN117892224B (en) Mixed characteristic neural network signal processing hardware circuit with wake-up mechanism
Feng et al. Research on Dynamic target Detection and Precise Positioning Based on PIXEL CODE YOLOv3 Deep Learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination